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A LEARNING MODEL TO IDENTIFY FACTORS CONTRIBUTING TO THE 
DIFFICULTY OF A PROBLEM ITEM WAS SUPPORTED EMPIRICALLY; AND 
INDICATED THA’*' THE NUMBER OF STEPS REQUIRED TO SOLVE A 
PROBLEM WAS TH MOST IMPORTANT VARIABLE IN PREDICTING BOTH 
ERROR PROBABILITY AND RESPONSE LATENCY. THE MODEL; IN ORDER 
TO ESTABLISH DIFFERENTIAL PREDICTIONS OF DIFFICULTY IN 
SOLVING ARITHMETIC PROBLEMS; IDENTIFIED SUCH VARIABLES AS THE 
MAGNITUDE OF THE LARGEST AND THE SMALLEST NUMBERS APPEARING; 
THE FORM OF '.HE EQUATION IN WHICH THE PROBLEMS ARE PRESENTED; 
AND THE DIFFERENCES BETWEEN NUMBERS IN SUBSTRACT IONS. ANOTHER 
VARIABLE; NUMBER OF STEPS (NSTEPS) RE JIRED TO SOLVE THE 
PROBLEMS; WAS FURTHER DIVIDED INTO TRANSFORMATION (STEFS 
REQUIRED TO PUT THE EQUATION INTO CANONICAL FORM); OPERATION 
(NUMBER ‘of OPERATIONS PERFORMED) ; AND MEMORY (NUMBER OF 
DIGITS THAT MUST BE HELD IN MCMORY) . TERMINALS WERE INSTALLED 
IN EIGHT CLASSROOMS; AND 270 THiRD; FOURTH; FIFTH; AND SIXTH 
GRADERS PARTICIPATED FOR ONE ACADEMIC YEAR IN 
COMPUTER-ASSISTED MATHEMATICS INSTRUCTION. A REGRESSION 
ANALYSIS SHOWED NSTEPS TO BE THE MOST IMPORTANT VARIABLE IN 
PREDICTING BOTH ERROR PROBABILITY AND RESPC^oE LATENCY; AND 
ANALYSIS OF THE FACTORS IN NSTEPS SHOWED THAT MEMORY WAS THE 
MOST IMPORTANT FACTOR. OPERATION PLAYED NO ROLE IN PREDICTING 
THE DEPENDENT VARIABLES. (OH) 
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1, Introduction . 

In the cognitive domain mathematics provides one of the clearest 
examples of complex learning and performance, for the structure of x-he 
subject itself provides numerous constraints on rny adequate theory. The 
learning and performance models derived from the main trends of contem- 
porsiry mathematical leai'ning theory have provided an excellent predictive 
analysis of a large variety of experimental situations. Unfortunately, 
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however, most of these e:jq)erimental situations are much simpler in struc- 
ture than what corresponds to even the simplest parts of elementary mathe- 
matics. Because this claim is central to the motivations behind the 
present paper, we would like to espand on it in some detail. 

The familiar and now classical, linear model provides a good starting 
point for our discussion. For our purposes, we may take this model in 
its simplest form, as applied to a situation in which a given response 
is always reinforced, and all other responses are indicated as incorrect. 
For the formulation under this restriction let probability 

of an incorrect response on trial n + 1. Tliis probabxlity is then the 
following simple linear function of the probability of an incorrect 
response on trial n: 



where the learning parameter O. is such that 0 < Of < 1. The formal 
properties of this simple model have nov- been investigated thoroughly and 
are well understood. It is apparent, hovrever, that if the subject must 
learn a number of different items which differ in structure and therefore 
in learning difficulty, the Simple linear model can accommodate this fact 
only by separately estimating a learning parameter OJ for each item. 

From the standpoint of classical psychological concerns with the character 
of learning and performance, this is far from satisfactory, What is 
desired, rather, is an analysis of the factors in the structure of the 
stimulus item which lead to varying difficulty. The estimation of a 
nonstructural parameter unique for each item is a way of handling data 
when no better resources are available, but it does not take us very 
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given domain of mathematics, we would like to be able to attach wei^tc. 
to the variou£j factors that may be ob^jectively identified in the item, 
and then use estimates of a few such weights to predict the relative 
difficulty or the latency of response for a large nunber of items. !Rie 
linear model itself cannot provide such mechanisms. This is not . to 
denigrate the inportance and significance of the linear model, for it 
will doubtless enter in many places to provide an analysis of particular 
mechanisms. But it will not serve as anything like the basis for a funda- 
mental or general theory of cor^lex learning . 

At first glance, it might appear that we could use a learning theory 
with more structure, such as stimulus- sampling theory, to provide an 
adequate analysis of stimulus structure- -adequate to make differential 
predictions of difficulty in cognitive domains like that of elementary 
mathematics. An examination of the ej^plicit axiomatizations of stimuj-us- 
sampling theory, which may be found in Estes and Suppes (l959)» Suppes 
and Atkinson (i960) or Atkinson and Estes (1963)> shows, however, that 
the concept of stimulus used does not provide an adequate analysis of 
struct\ire. Roughly speaking, the situation is the following. The stimuli 
presented to a subject on a given trial are represented by a set of stimulus 
elements. In the concept of an arbitrary set of stimulus elements, there 
is the beginning of an adequate apparatus for the concept of structure, 
but the additional assuiooptions ..eeded for a definite notion of structure 



deeply into the psychological problems of learning complex 
those common in mathematics and other structured subjects, 
the estimation of a separate parameter for each item leads 
use of parameters. In general, if we take a collection of 
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have not yet been in?)Osed on the concept of an arbitrary set. It is 
necessary to go beyond the current formulations of the theory in order 
to analyze even the simplest sort of stimulus items used in the. teaching 
of elementary mathematics. Probably the most successful version of 
stimulus" sampling theory for a wide variety of experiments is the pattern 
conception of stimulus conditioning that originates with Estes (1959)* 

On this theory, the individual stimulus elements are not conditioned as 
consonants to a correct response, rather, an entire pattern of stimulus 
elements is so conditioned, and in general the nuniber of patterns available 
for sanpling in a given stimulus situation will be a parameter to be esti- 
mated from the data. But even these conceptions are very far from providing 
an analysis sufficiently structured to yield differential predictions of 
difficulty in responding correctly to problem-items drawn from concepts 
and topics in elementary mathenatics. 

It might also be thought that the applications of stimulus- sampling 
theory or related sorts of theories to stimulus-discrimination problems 
during the past decade would yield theoretical ideas adequate to the 
analysis of conplex structure. Again, however, an examination of the 
kinds of problems that have been handled shows very quickly that a struc- 
tural apparatus adequate to problems in simple addition, for exanpie, is 
certainly not even implicitly inherent in the theories that have been 
developed within the general framework of theories of conditioning. 

Both psychologists and educators interested in cognitive theories in 
learning would undoubtedly very much agree with the remarks we have just 
made about stimulus-response theories. However, we find that vze must say 
the same sorts of things about the current cognitive theories of learning 
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and performance, whiclTi have attracted considerahXa interest ^n the last 
few years. As opposed to the stimulus- re sponge theories .that .?e have 
mentioned, perhaps the greatest defect of the cognitive theories is simply 
a lack of sufficient intellectual definiteness even to settle the question 
of whether or not specific predictions can be made. The kinds of cognitive 
considerations, for example, that enter into the studies reported in the 
well-known book by Bruner, Goodnow and Austin (1956) simply do not provide 
a framework vrithin which we can ask specific questions about the estimation 
of parameters for the prediction of differential difficulty over a selection 
of stimulus items drawn from some complex domain, whether it be elementary 
mathematics or elementary language learning. Again we woula not want to 
be misunderstood on this point. The analysis of. the types o- strategies 
used in concept attainment is uer.'tainly a useful contribution to the 
psychology of concept formation and thinking, but it must be realistically 
asserted that no theory has yet been sufficiently developed to provide 
the kind of parametric predictions that are considered a minimum require- 
ment in the aj.'ea of mathematical models of learning and performance. The 
same sorts of remarks apply to the invaluable work of Piaget and his col- 
laborators. Pdaget has contributed much to our understanding of cognitive 
development in children and especially to our understanding of the kind 
of structures children find or if you wish, create, in the stimulus envi- 
ronment. But again, Piaget* s concepts have not been sufficiently artic- 
ulated into a well-defined theory to provide parametric predictions of 
differential difficulty for items drawn from any cognitive domain. This 
is not particular]/ Piaget’s task, as it was not Bruner’s. Nevertheless, 
we do intend our remarks to be of a critical nature, for until parametric 



predictions can be produced from the theoretical prq[)Osals generated by 
various psychologistsj these theoretical ideas cannot be accepted as a 
final analysis of what we hope to understand about cognitive processes » 

The preceding remarks have mainly einphasized the inadequacy of current 
psychological theories to provide parametric predictions of differential 
difficulty as measured by the rate of correct responding. These theories 
are evjn more inadequate when we turn to response latencies* From the 
standpoint of the analysis of performance, latencies are in many respects 
more important as a source of information to the theorist then response 
data. This is particularly true of any studies devoted to skill per- • 
formance after a good deal of learning has taken place. As soL.e of the 
data reported here show, and ap one would expect anyway on a priori grounds, 
the range of latencies observed in a group shows systematic variation in 
a way that cleai'ly reflects a measure of item difficulty* .What is ulti- 
mately desired in this case is the ld.nd of. model that can predict from the 
structure of an item ^he process a subject must go through in finding tha 
• correct response* In the case of arithmetic, at least part of this process 
must undoubtedly he related to the standard algorithms taught as part of 
the curriculum; hut even a casual glance at these algorithms will show that 
the conception of them used in teaching and in the curriculum does not pro- 
vide a sufficient analysis of processing to make differential predictions 
of difficulty as reflected in response latencies* 

What is also surprising about latency is that there have been so few 
studies that reported detailed data on this measure. The only directly 
relevant studies that we have found in the literature on arithmetic are 
Batson and Comhellick ( 1925 ), Helseth ( 192 ?), Knight and Behrens ( 1928 ) 
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and Billington (l^7)» absence of latency studies (even tbougi^ there 

are undoubtedly several of \»bich ve are not aware) indicates hew superficial 
has been the investigation, of structural models adequate to predict differ- 
ential difficulty either in terms of responses or response latencies* 

The constructive aim of the present paper is to formulate ar;,d test 
some linear structural models that do lead to parametric predictions of 
the sort we have been discussing. The sense in which these !MOdels are 
linear is not precisely the same sense that applies to the linear learning 
model; but it is in the context of linear-regression models, a point that 

/ 

is made clear in the next section. The models and accon 5 >anying theory 
which we present and test in this paper are meant only" as a beginning. 

We do believe that they provide a significant and. premising foundation 
for further work. 

2 . The Theory . 

The learning models that arise in stimulus- sanpling theory all 
exemplify a certain class of stochastic processes, and in general a dif- 
ferent class of such processes is exemplified by the linear models discussed 
at the beginning. In the same fashion, the linear structural models pro- 
posed in this paper all exemplify a general class of models that are clas- 
sical in statistics. But simply to say that we are applying linear-regression 
models to the study of arithmetic performance provides no more clue to the 
theoretical ideas behind the analysis than does the assertion that we apply 
to a given body of learning phenomena a finite state Markov chain as the 
primary mathematical tool of analysis. What is important and significant 
for psychology is the particular way in which the broad class of 
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line®r-regr6ssioii models is narrowed and made? meaningful from the stand- 
point of response or latency performance in arithmetic. 

It will perhaps be useful to begin with a class of problems that are 
siH 5 >ler than those considered here in detail. The discussion of this first 
exan 5 >le follows Suppes (1966). Let us suppose that a set of problems con- 
sists only of simple addition problems of the following sort: 1 + 2 = n, 

1 + n 3 and n + 2 =3. Let us restrict the sums to those not greater 
than b . We postul.ate that the following five facts ere held in memory; 

!.==•/ 

2 =// 

3=/// 

5 = ///// . 

Our algorithm is then the following: 

(1) Replace all Arabic numerals by their stroke definitions md 

delete all plus symbols. 

(2) If there are strokes on both sides of the equal sign, cancel one 
by one, starting from the left of each side until there remain no strokes 

on one side. Ignore n in cancelling. 

(3) On the one side, still having strokes, replace the strokes by an 
Arabic numeral, using the definitions in memory. 

The solution in the form n = c or c - n will result. 

To obtain a single factor f representing the number of steps, we simply 
count the number of steps required by the algorithm to solve a given problem 
For example, the steps to solve 3 + n = 5 are 5 in number. 
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rule 
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rule 
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and thus for this model and this problem, f = 5* A more realistic version 
of this algorithm, at least for many standard situations ^n which students 
are tested on their command of the sin5>iO addition facts, is' to postulate 
that the student counts the difference n, by beginning at 3 aiid stopping 
at 5 • A test of five variants of this latter counting algorithm is 
reported in Suppes and Groen (1966) . 

For the problem-items analyzed in this paper the central problem is 
to identify the factors that contribute to the difficulty of the Item. 
Typical factors that we shall examine are the magnitude of the D.argest 
number appearing in the problem, the magnitude of the, smallest number, 
the form of the equation in which the problem is presented, and most 
importantly, the number of steps required to solve the problem. Exactly 
how the number of steps is to be defined is a matter that we take up in 
detail below. As a matter of notation we shall denote the j ''' factor 
of problem 1 in a given set of problems or exercises by f^ The 

statistical parameters that must be estimated from the data are the 
weights to be attached to each factor. We shall denote the weight 
assigned to the j factor by . We want to emphasize as explicitly 
as possible that, the factors identified an.d used in the models presented 
in this paper are never factors in the sense of factor aj,ialysis; that is, 




the factors do not arise as abstract constructions from the data. Rather , 
they are always objective factors identifiable by the experimenter in the 
problem- items themselves, independent of any data analysis. Which factors 
turn out to be inportant is a matter of the estimated weights Oty but in 
no case does the decision as to what is the numerical value of a factor 
for a given problem-item depend in any way on the response data themselves. 

In fact, it will be apparent that all of the factors used in the analyses 
presented here have an intuitive and direct relevance to commonsense ideas 
of difficulty, and their definitions are so straightforward and simple chat 
there is little .prospect of disagreement over their objective value in- a 
given problem- item. 

We may first consider the analysis of response data. Let p^ be the 
observed proportion of correct responses on problem-item i for a given 
group of subjects. The central task of a. model is to predict .the observed 
proportions p^. The natural linear-regression node! in terms of the 
factors f^^ and the weights is simply 

, Pi = 5 “/id * “o • 

However, there is a central difficulty with this particular model: there 

is no guarantee that probability will be preserved as the estimated 
weightings and identifiable factors are combined to predict the observed 
proportion of correct responses in new items. Consequently, in order to 
guarantee preservation of probability, that is, to ensure that the predicted 
p.*s will always lie between 0 and 1, it is natural to make the following 

i 

transformation and to define a new variable 
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( 1 ) 




j^d then to US© as the regresfiion model 



= E o/ij * «o • . 

It shoxild be noted that the reason for putting 1 - ra’tjher than p^ 
in the numerator of equation (l) is that it is desirable to make the vari- 
ables monotonically increasing in the magnitude of th- factors f^^ 
rather th^ monotonically decreasing. For example, the magnitude of the 
largest number in a problem increases with the difficulty of the problem, 
and it is desirable that the model reflect this increase in a direct rather 
than in an invc»rse fashion. 

In the case of latencies a transformation like (l) is not required. 

Let t^ be the mean latency on problem-item i for a. given group of 
subjects. We then apply the same model as (2), namely. 



t. 

a. 



= E 

3 






( 3 ) 



It is also evident that no transformation is required to make latencies 

monotonically increasing in the ejipected difficulty of the factors. We 

have shown different weights p. for the latencies, because the empirical 

J 

% , 

interpretation of the weights must necessarily be different for the variables 
z^, but as we would expect, there is a high positive correlation between the 
weights a. and p,. It is worth noting that in the case of the analysis 

u u 
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of the latencies, -ohe individual factors and their weights may be identified 

as t^e direct contribution of a given factor to the total latency. Thus, 

for exaiople, the contribution of factor j to the totaX latency is just 

the number 6,f. . which is scaled in seconds. The constant that arises 
j 

in the linear- regression model may be interpreted as the constant orienta- 
tion and preparatiCMi time required in solving the problems of the class 
under investigation. 

The variables we consider are of two sorts. The first is the Mnd of 

0,1 - variable standard in the analysis of variance. Such a variable would 

\ 

be appropriate, for exanple, in dealing with problem format, ^ne second 
kind of variable is one that is in principle continuous, although in prac- 
tice it assumes a finite set of values for the problems being considered 

\ 

here. For most of these variables the conception and formal definitions 
of the variables are quite straightforward within the context of elementary 
arithmetic itself. Typical variables have already been mentioned; however, 
the variable or factor dealing with the number of steps required to solve 
a problem is most in^ortant from the standpoint of the psychological 
analysis, Tlriis factor also seems most promising for future developments 
of the models presented in this paper. We turn now to the appropriate 
formal definitions. As has already been en 5 >hasized, we feel that the 
greatest possibilities for subseq;aent theoretical analysis lie in this 
direction. What we report here is only the result of our first relatively 
crude analysis, and we are already heavily engaged in the process of 
deepening this an^ysis, particularly by breaking up the single variable 
of number of steps into several components. Some preliminary results are 
reported at the end of the paper. 
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The steps postulated have been broken up into three classes: those 

required to transform the problem into canonical form, those corresponding 
to the number of operations performed, and those corresponding to the 
number of digits that must be held in memory. We refer to these three 
classes as the transf ormation , operation , and memory classes. As will 
be seen, there is a quite hi^ correlation in most problems between the 
number of operation steps and the numiber of memory steps. An essential 
point for later work is to make these two processes more orthogonal in 
operattoral characterijsation. Another assumption that is sxarely too simple 
is reflected in the assignment of the same weight to addition and subtrac- 
tion, in the analysis of operation steps. Other ainrealistic simplifications 

have been made, but the general definitions required to characterize the 

\ 

number of steps required for solution are still relatively complex, and 
we think they constitute a reasonable beginning. 

For simplicity we first consider just the transformation steps that 
convert any problem into canonical form. By canonical form we mean the 
equational form in which the blank or unknown stands by itself as the only 
term to the right of the equal sign. Thus for numbers m, n and p, 
regardless of whether the numbers are one digit or two digit, we have 

(i) m + n=s is already in canonical form, 

(ii) m + = p is transformed to = p - 

which is transformed to p - m = , requiring two steps, 

(iii) + n - p is identical to (ii) 

(iv) m “ = p is transformed to m - p = , requiring one step, 

and finally 
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(v) - n p is transformed to n + p = , also requiring 

one step. The fact that m + __ = p requires one more transformation 

than m “ ~ P agrees with the intuition that (ii) is reall y more 

difficult than (iv). We make, explicit the number T of transformations 

in the following definitions that -formalize (i) - (v). 

T(m + n = ) = 0 

T(m + = p) =2 

T( + n = p) - 2 

T(m - = p) = 1 

T( - n - p) = 1 

T(m + n = p + ) =s 1 

T(m + n = + p)-l‘ 

The last two equations cover two additional cases that arise in the data 
we analyze. 

Turning now to the operation and memory steps, \ie need to make 
63 qplicit the number of digits involved, so we always use initial letoers 
of the alphabet for single digits. Also, because we postulate that the 
operation and memory steps enter only ajfter the transformation to 
canonical form has taken place, we may simplify the notation, writing, 
for example, 0(ab + cd) or M(ab + cd) for the number of operation 

or memory steps respectively. For example, 

0(5 + 0 ) =: 0 

but 

0(5 + = 1 

because we postulate no operation is required for handling zero. 

0(15 + jJ2) = 2 . 

N 

because one operation is 5+2 and the second is 1 + 1. On the other 
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hand, in the form ab + cd, when b + d > 9j 'there are three operations. 



For example, 



0(25 + 47) = 3 . 



because one operation is 5 + 7j the second is the partial sum 1 ? 

using the 1 that is "carried"; and the third is 3 + the partial sum 
plus 4 , the other tens ' digit . 

In the case of mempry, 

M(15 + 12) = 1 , 

because only 7j the sum of 5 2, must be held in memory while the 

tens are added and the correct tens' digit response is rade (the problem 
format required input of the tens’ digit before the ones' digit). On the 
other hand, 

M(25 + 47) = 3 

because (i) tne 2 of 12, the sum of 7 5j must be held in memory 

for the ones' response, (ii) the 1 which is carried to the tens' place 
laust be held, and (iii) ’•.he partial sum 1+2 must be held while it is 
added to 4. The definition for the more complicated format ab + cd - ef 
is given recursively in terms of ab + cd, and thus does not need a 
separate treaument. Formally the definitions of the number of operation 
and memory steps are as follows: 



G if a = 0 or b = 0 



0(a + b) = i 



1 if a?^0&b^0 



0(ab + d) = 



0(b + d) if b + d < 9 
0(b + d) + 1 if b + d > 9 
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0(ab. + cd) = 



< 



0(a - I5) = 
0(ab - c) s 

0(eh - cd) = 

M(a + b) = 
M(ab + c) = 

M(ab + cd) = 

M(ab - c) = 

M(ab - cd) = 

If ab + cd = gh then 

0(ab + cd - 



0(b + d) + 1 if b + d < 9 
0(b + d) + 2 if b + d > 9 

I 0 if b = 0 
if Id ^ 0 

0(b - c) if b > c 

-c)+l if b<c 



1 


if 


0 

n 


2 


if 


b > d > 0 


3 


if 


,b < d 


0 








if 


b + c < 9 


1 ' 


if 


b + c > 9 


r 

1 


if 


b + d < 9 


i 

3 

> 


if 


b . d > 9 




if 


b > c 




if 


b < c 




if 


b >'d 


|3 


if 


b < d . 



ef) 0(ab + cd) + 0(gh - ef) 





and 



M(ab + cd - ef) = M(ab + cd) + M(gh - ef ) + 1 » 

The additional step in the case of M(ab cd - ef) ccames in from having 
to remember a + c, or a + c+1, as the case may be, which is not part 
of M(ab *r cd) or M(gh - ef). 

Similarly, if ab + c = f g then 

0(ab -t* c - de) - 0(ab + c) + 0(fg - de) 

and 

M(ab + c - de) = M(ab + c) + M(fg - de) . 

A ccjrre spending definition holds for ab cd - e. 

In evaluating problem structure, we determine the total number of 
transformation, operation, and memory steps. Thus, for example, 

25 + 26 = 18 + _ 

has the maximum number of steps, because 

T(25 + 26 = l8.+ __) = 1 
0(25 +26-18) =6 

M(25 f 26 - 18) = 7 , 

and on the other hand the problem 

5 + 0 = 

has the minimum of 0 steps. Of course, some students will solve many 
individual problems by a shorter method, and the present approach to 
counting steps does not incorporate emy such special methods. This again 
is a matter for subsequent investiga/cion. 

In the analyses reported in this paper we have entered the total 
munber N of steps as a single variable for most of the results reported, 
but in one case we have broken the steps up into classes, and further 
intensive work in this direction is currently underway. In the linear- 
regression models used for this purpose, we replace oN by a^T + o^O + a^M . 
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3. Methods 



The data reported and analyzed in this paper were collected as an 
integral part of a full academic »-year, operational prog'^'am in cominiter- 
assisted mathematics instruction. For this reason we shall describe xn 
some detail this program; 

Subjects . The approximately 9:j0 subjects in this project consisted 
of the entire population of grades three , fou; y five , and six in the Grant 
llementary School, except for those in the handicapped classes. 

The cnlldren came from a middle- class, suburban community. All 
children lived within walking distance of the schaol. 

Although there was some fluctuation in atten<^aa.ce figures during 
the year, school records show the following population figures at year’s 
end. There were 32 boys and 30 girls in three, 4l boys and 35 girls 

in grade four, and 44 boys and 26 girls in grade five. The mean I.Q. pf 
the fifth-grade group was ll4,, the range 72-145* Grade six had 35 boys 
and 27 girls. Mean I.Q. of the siaith-grade class was II 7 , range 88-156. 
There were no data on I.Q. scores for either grade tlriree or grade four. 

Equipment . The student terminals used in tliis project were commer- 
cially available teletype machines, connected by private, high-speed, 
phone lines to the Institute’s computer at Stanford. A large book closet, 
which opened inxo the classroom, was modified by adding a vx,..tilation 
fan, light, and e‘> •'trical cutlet. This provided privacy for the user 
and insulated the rest of the class from the operational noise of the 
teletype. 

The control functions for the entire system were handled by a medium- 
sized computer. The PDP-1 has a l6,000 word core, and a 4,000 word core 

l3 



o 




which can he interchanged with any of 32 bands of a magnetic drum. All 
input-output devices are processed through a time-sharing system. Two 
high-speed data channels permit simultaneous computation and servicing 
of peripheral devices. Additional backup in compucational power, additional 
storage, and increased input-output speed are obtained through connections 
to disk storage of a larger computer (X£M 7090) located st the Stanford 
Computation Center. 

Response time was measured from the instant (neai^st ,001 c^ec.) the 
type wheel was in position at the resporije area (o> answer blank)# ^Hen 
the student depressed one of the keys on the teletype's keyboard, a. signal 
was so nt to one computer. The character was recogrdzed by the computer 
approximately one millisecond (.001 sec.) after being ini+iat^'d by the 
student, A reading was taken from a r* vil-time clock, internal to the 
CMiputer, and this information compared to the tinie read when the type 
wheel was positioned. Under optimal conditions latency measurements 
could be .niade with an accuracy of from two to three milliseconds. How- 
ever, as mentioned above, the system was operating under a time-sharing 
arrangement. This reduced the le'/sl of accuracy of latency measure to 
about one-tenth of a secohd. Conversion from readings in thousandths to 
the nearest tenth was made by division and truncation. 

Curriculum Materials « Daily lessons were prepared and organized by 
concepts or topics into blocks or units. The concept blocks were arranged 
sequentially corresponding appioximately to the order of topics in the 
textbook series. Sets and Numbers , written by the first author of this 
paper, length of time needed to complete a concept block varied 

from three to twel’.'e days, when a single lesson was taken each day. The 
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curriculum objective of the dally lessons "Was to provide an organized 
program of review, maintenance and drill on basic skills and concepts of 
elementary mathematics, pait:icularly arithmetic*. Initial instruction in 
all concepts was given initially by the teacher, and consequently the 
drill-and“practice work at computer temdnals did not include a detailed 
introduction to the concepts. 

Teachers involved in the project were free, subject to certain 
eohStralnts, to select any of the prepared blocks ih order to correlate 
the drill-and“practice work with their daily instruction. Handbooks were 
furnished i^ch described available concept blocks in detail. Also 
included in the teandbooks were reprints of every lesson. Table 1 describe? 
the concept blocks prepared and planned for each grade level. 

Insert Table 1 alx>ut here 

Each concept block was oi'ganized in the manner shown diagrammatically 
by Figure 3.. Lessons were prepared at each of five levels of difficulty 
within each concept block. Among factors which determined intuitive 
estimates of relative difficulty are those discussed in this paper. 

They, and the exercises reflecting them, were chosen intuitively on the 
basis of teaching experience and previous project e:jqperience gained from 
preparation and testing of the textbook series cited above. Each class 
was restricted to a single concept block at a time. On the first uay 
of a new block, evecty member of a class was given, the same lesson. This 
lesson was of average difficulty (level 3)- ^^ose students who scored 
between 60^ and 79^ were given a level- three lesson the following day; 
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Insert Figure 1 about here 



tiiOGS who scored above YS^b we;re given a lesson on the uext higher level 
(level k); those who faiD.ed to score at least 60# vere given a simpler 
lesson on a lower level (level 2). This procedure was followed through- 
out a concept block, that is, a score of above 79^ branched a student up 
one level each day, while a score of below' 60 ^ branched a student down 
one level each day, but of course a student could not move up beyond 
level 5 or down below level 1. Thus, by day three, a student could have 
been at any one of five levels , with a different lesson at each level. 

It was intended that approximately 9^^ of the students would rltemate 
between levels two, three, and four, and that those remaining on ai^ 
level would be nearly homogeneous. Level 1 was mainly remedial in char- 
acter, and level 5 was ordinarily meant to be difficult. Drills on all 
levels increased somewhat in difficulty from day to day within a block 
as successively more advanced aspects of each topic were reviewed. 

Progreim logi c . Under computer control each problem was completely 
typed out, including a blank for the response. The type , wheel of the 
teletype was then positionid at the blank so that the response would be 
properly placed® A correct response was reinforced by the appearance 
of the next exercise. When an incorrect first response was mads, the 
word "wrong" was typed out and the exercise itself was retyped. A second 
error on the same exercise was followed by the message "wrong, the answer 
is ", with the correct answer being displayed. The exercise itself 
was then retyped once more to allow for a correction response. An error 
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on the correction response caused the correct answer to he given again.> 

1 

but whether the third response was Correct or incorrect, the next ^ercise 
was presented* 

If a response was not given within a predetermined intervcd of time, 
usually ten seconds, the machine response followed the above pattern except 
that the words ”time is up” were substituted for the word ” wrong” at each 
step described above, k flow chart of the program logic is given in 
Figure 2 . 

«p» wm m «j> M mm w mm mm w mrm w mm mm ^ mm ^ mm m mm w w 

Insert Figure 2 about here 

Procedure , The two classes in each of grades four, five, and six 
began in October, I965 sharing one teletype between them. One class- was 
scheduled to run in the morning, the other in the afternoon. However, 
this proved to be an unworkable arrangement. Beginning with the third 
week, the classes worked on the teletype on alternate days. The machines 
were operated dally between the houis of 8:30 A,M, and 3 -JOO P.M. 

Ill late February, i960, the twc third-grade classes began daily lessons 
with the addition of two more teletypes, \diich brought the total number of 
machines in operation on a daily basis to five. In early April, I966, the 
last three machines were put in operation, bringing the total number of 
teletypes to eight. Each class in grades three, five, and six had its 
own teletype. Grade four had been divided into three classes to alleviate 
an overcrowded situation. One of the fourth-grade classes had its own 
machine, the other two classes sheired the remaining teletype. 

The students took their lessons one at a time on each machine in the 
order prescribed by their teacher. The program began by asking the student 
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Figure 1. Diagram of "branching str^ctiire followed in constructing sets 
of exercises for concept "blocks. 
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to type his name. When the name had been correctly input, the lesson 
began as described above in the section on prograiri logic. If a student 
failed to spell his name correctly, or gave a fictitious name (such as 
Batman ) . the program asked him. to try again, An individual history was 
kept in computer memory for each student. When a student* s name was input 
correctly, the proper ].esson was selected, based on the branching criteria, 
and presented automatically. Students were free to sign on at any one of 
the machines in the school at any time during the day. It was also possible 
to take more than one lesson a day. 

Lessons were designed to take from four to six minutes each, with 
an average of about five minutes, to allow each student in a class to take 
one lesson each day. The usual number of problems per lesson was twenty. 



Follovdng the j.c'ssou, a summary of the student *s work was given, A sauiple 
print-out of a lesson taken by a fifth-grade student,' Mike O’Dell, is 
given in Table 2. The numbers given in the summary for correct, wrong, 
and time-outs are for first response only. The nmbers following the 



Insert Table 2 about here 



word "wron^ are problem numbers. As shown., Mike had 8156. correct in this 
concept block, 595^ correct to date for the whole school year, which began 
October 18 on the teletype. The time given in ho\irs, minutes, and secoi 
is the total time Mike had spent on all teletype lessons to date. 

The students were not allowed to use pencil or paper when working 
on the teletype,. Each exercise was worked on the machine so that all 
responses could be recorded and latencies measured. The response mode 
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Tatle 2o Sample print out of a flftli-grade student. 



m.cMc im YOUR MAna 



RIKC OtlMl 



•fSIU. NtMtCX 909013 



(4«| ♦ 03> / V • (<fi / ^1 * (03 / 



43 • 33 • 33 

WKM33 



43 • 38 • 33 <• ul. 
tNtONO. ANSKtR S3~M 



43 • S3 • 33 • ^ 

10 - <8$ - fO> ■ (TO • 80) « 

4 X (1 ♦ 13> • (4 X ♦ (4 X 

(93 • 80> • 11 ■ S3 • (80 ♦ 

33 ♦ (74 ♦ 18> • (^ * 74> ♦ 13 

91 X (30 X 18> *> X 30> X 18 

IT X (S4 ♦ S4> » (IT X !4> * (IT X ^1 
348 ♦ 943 • 943 ♦ 

(9 ♦ ft> X T ■ (^ X 7> ♦ (-I-- X 
(90 / tO> / 3 • ^ / (to X 3> 

(Ti / 9> / 4 ■ Tt / (^ X 4> 

_> 



(94 ♦ 13 > / 0 ■ (94 / 0 > ♦ (to / 
TIME 13 'ir 



(94 ♦ 13> / 0 ci (94 / 01 ♦ (U ^ 
30 • (19 - 18> -v (00 • * li 
Tt X (43 X tt> ■ (93 X 43 > X ^ 



(03 / 7> ♦ C90 / 7> • (=4* ♦ X 7 

¥sm% . 



(OS / T> ♦ (99 / n ■ -=||> ^ ^ 



IN> Of BXSL&. HUN 3 EX S 09013 
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was limited to either n\imerical answers or siiaple single-letter answers 
for raultiplv^- choice problems. 

Znitia!'- instruction on the teletype and program operation consisted 
of explaining bo each class the general procedure of taking turns on the 
machine, and of showing that onl;y the answer need be input on the keyboard. 
The program logic was also eacplained. Staff memibers helped each student 
find the letters to type his name for the first two or three lessons. 
Students had little trouble learning how to type their names or answer 
the questions. 

Following the summary and "goodbye” message the student was told 
"please tear off on dotted line". A dotted line was printed, and the 
student then tore off his print-cut and took it with him as a permanent 
record of his work. 

If. Results . 

To begin with, it must be en?)hasized that we have not attempted a 
detailed model- theoretic analysis of data from all the concept blocks 
listed in Table 1. We have selected five topics on which we had consid- 
erable data and which were sufficiently simple to provide a good starting 
point. The first analysis- deals with fourth-grade and fifth-grade per- 
formance on addition, the data are drawn from blocks 1 and 3 of grade 
four and block 1 of grade five listed in Table 1. The second analysis 
is concerned with subtraction at the same grade levels; the data are 
drawn from block 2 for each grade. The third analysis looks at fourth- 
grade multiplication data, drawn from block 5 • The fourth- analysis deals 
with a relatively- controlled experiment on the multiplication tables for 

2lf 



grades 3-6; the data are d?.*awn from blocks 37 > 35 s 3i> respectively, 

fof each, 'of grades 3-6. The final analysis returns to the data of the ; 
first analysis and looks at the results of breaking up the regression 
analysis of the number of steps into several variables, as indicated in 
the theoretical discussion. 

As remarked earlier, for each set of problems examined success-latency 
and error probability have been treated as separate dependent variables. 
Separate regression coefficients were obtained from the same independent 
variables to predict latency 8m.d error probability. This is justifiable 
by the intuitive assumption that success-latency and error probability are 
different measures of common underlying processes, and is justified empir- 
ically by our finding that the correlation between the two dependent 
variables was consistently greater than 0.7 for the data we have collected. 

To minimize the effects of subject variables such as I.Q.> the 
problems and data were usually treated separately by grade, concept block 
and level, as is made explicit in the tables given below. It is assumed 
that children working within a given branching level form a more homogeneous 
group of subjects than children working on different level problems. We 
were unable to analyze data from some of the levels available because too 

few children entered those branches. 

Tlie first step in analysis was to obtain regression coefficients for 
each grade and level for the two dependent variables. A stepwise, multiple 
linear-regression analysis program, BIMD 02R> adapted for Stanford Uni- 
versity's IBM 709 c computer, was used to obtain regression coefficients, 
multiple correlation R ■ and R^. For a finer-grained analysis of the 
goodness of fit of the success-latency predicted from the regression model 
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and observed success-latency, a computer progrsm was written to calculate 
the predicted mean success-latency for each probj.ea and to give as a 
measure of fit 



S2 = 



N o 

(obtained latency^ - predicted latency^) 



N - k 



where N is the number- of problems for which the latency was predicoed 
and k is the number of estimated parameters. Similarly, for a finer 
analysis of the goodness of fit of the regression model to the error data, 
a program was written to calculate the predicted proportion of errors for 



each problem i from the obtained regression coefficients and to give as 



a measure of fit X^, where 



2 (fi - PiN) 
= p^(l - 



and 



I 




f^ = observed frequency of correct responses, 

p = predicted probability of a correct response-, 
i 

N = number of students. • 

Addition - - grades four and f ive . The three independent variables used 
in the regression analyses for addition were the variable NSTEPS, which 
was described in detail earlier, and the/two magnitude variables, magnitude 
of sum (MGSUM) , and magnitude of the smallest addend (MAGSMALL) . It is 
obvious that the value of MAGSbM and MAGSMALL is independent of whether 
the problem for the student was to find the missing sum or a missing 
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addend. For example, in the three related problems 1 + 9 - » 

7 + = l6 and + 9 = l6, MAGSUM = l6 and MAGSMALL = 7 • 

The coefficients obtained for the regression equations are 
shown in Table 3 > This table indicates the level of problems ana- 
lyzed, (level), the number of children who worked on the problems 

Insert Table 3 about here 

in that level, (Subjects)^, the number of different problems ana- 
lyzed, (Problems)^, the regression constant and the regression 
coefficients for the three independent variables. The absence of 
a value of a given coefficient indicates that the variable it applies 
to made no significant contribution to the regression equation, and 
the conputer program therefore did not use that variable in obtaining 
a regression line. In reading the regression table it should be 
remembered that the transformation described previously was applied 
to the observed proportion of errors, and therefore when obtaining 
a prf;diction from the coefficients for proportion of errors, the 
numbers z^ calculated from the coefficients must be transformed 
to obtain the predicted proportion of errors. 

It is clear from scanning the coefficients in Table 3 that NSTEPS 
is the most important of the three variables in predicting both errors 
and success- latencies . A rough indication of the goodness of fit of the 
regression lines is reflected by the multiple- correlation coefficient R 
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Table 3 

Linear-regression coefficionts for fourth- and fifth-grade addition 
Grade k Addition, Set I, Proportion of Errors 









1 

M 1 . . 1 - 




UiTJi/^rrr " 




"D 


.2 


Level 


subjects 


Problems 


uonsuano 


iMOXiZiro 


rrLriUQwrx 








2 


6 


19 


-2.73 


0.16 


0.09 


‘-0.03. 


0.61 


0.37 


3 


^1 


38 


-2.65 


0.16 


0.05 


-0.C3 


0.56 


0.37 


4 


24 


38 


-1.44 


0.24 


-0.01 


0.05 


0,86 


0.74 


5 


9 


19 


-1.74 


0.08 


0.03 


-0.09 


0.40 


0.16 



Level 

2 

3 



Grade k Addition, Set I, Success Latency- 



Level 


Subjects 


Problems 


Constant 


NSTEPS 


MAGSUM 


MAGSMALL 


R 


R 


2 


6 


19 


0.24 


0.18 


0.14 


-0.07 


0.64 


0.40 


3 


’21 


38 


-0.76 


0.47 


0.13 


-0,09 


0.69 


0.48 


4 


24 


38 


2.32 


0.57 


-0.02 


0.07 


0.86 


0.7^^ 


5 


9 


19 


2.19 


0.17 


0.00 


0.00 


0,44 


0.19 



Grade k Addition, Set 2, Proportion of Errors 

Subjects Problems Constant NSTEPS MAGSUM MAGSM^LL 

7 57 - 1.69 0 « 1 T 0.02 - 0.02 

kl 95 - 0.73 0.21 - 0.01 - 0.01 

3l^ 76 -1.60 .0.20 0.00 0.01 



o.5i^ 

0 . 6 ^ 

0.80 



Grade 4 Addition, Set 2, Success Latency 
Level Subjects Problems Constant NSTEPS MAGSUM MAGSMALL 

2 

3 

k 



R 



R 

0.29 

0.41 

0.64 



7 


57 


0.95 


0.56 


0.c6 


-0.09 


0.64 


0.42 


41 


95 


1.77 


0.73 


0.01 


-0,06 


. 0.82 


0.68 


34 


76 


1.55 


0.47 


0.01 


0.02 


0.75 


0.56 



Grade 5 Addition, Proportion of Errors 
Level Subjects Problems Constant NSTEPS MAGSUM MAGSMALL R 



3 and 4 
combined 



xd 



57 



-2.41 0.10 0.03 



0.03 0.81 0.b6 



Grade’ 5 Addition, Success Latency 
Level Subjects Problems Constant NSTEPS MAGSUM B4AGSMALL R 



3 and 4 
combined 12 



57 



- 2.22 



47 



0.09 



R 



0.07 0.73 0.54 



and its square {!?) which is an estimate of the amoiint of variance 
accounted for by the regression model. In only one case is less than 

of the success-latency variance accounted for by the model,. When one 
tsdces into account that the two magnitude variables account for a 
relatively small amount of the variance^ and that in setting up the variable 
NSTEPS we have combined several potentially powerful and probably inde- 
pendent variables, the results are encouraging, 

p 

Tables 4-10 present the X and individual contributions of the 

p 

problems to X when the seven sets of coefficients for response errors 
given in Table 3 were used to predict the proportions of errors. Included 
in these tables are the rank order of observed problem difficulty the 
observed proportion of students making errors, (Observed (l - "fc^e 

proportion of errors predicted from the line 6U'-r egression model, (Predicted 
(l - p^)); and the actual component of the X contributed by the problem. 

Insert Tables t, 6,7> 9> 10 about here 

Table 11 presents the same analysis for fifth-grade addition* Unfortu- 
nately the data on addition for the fifth*-grade children are rather sparse. 

Insert Table 11 about here 

The fifth graders were presented with a concept block on addition during 
the first week of operation of the computer-based system. Technical 
difficulties and initial student unfamiliarity with the teletypes caused 
us to lose a good part of the data we had hoped to collect. 
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Table 4 



Predicted and observed proportions of errors and success -latency in 
fourth-grade addition, concept block 1, level 2 



Rank 


Equations 


Observed 
(1 - Pj^) 


Predicted 
(1 - Pi) 


Observed 

Latency 


Predicted 

I.atency 




1 


24 + = 24 


0,08 


0.21 


2.4c 


2.65 


0.61 


2 


’T ^ 


0*17 


0.44 


4.50 


4.05 


1.77 


3 


+ 0 -'t 21 


0.17 


0.22 


4.70 


3.46 


0.10 


k 


26 + = 26 


0.17 


0.26 


3.00 


2.78 


0.28 


5 


CVJ 

II 

o 

+ 

! 


0.17 


0.30 


3.00 


3.73 


0.49 


6 


26 + 2 = 


0.33 


0.50 


3.70 


4.26 


0.71 


7 


26 = 27 


0.33 


0.30 


2.50 


2.92 


0.02 


8 


23 + = 26 


0.33 


0.31 


2.40 


3.00 


0.02 


9 


22 + = 30 


0.50 


0.69 


5.50 


3.98 


0.98 


10 


21+7 = _ 


0.50 


0.42 


4.80 


3.90 


0.17 


11 


ON 

CVJ 

11 

1 

+ 

CVJ 


C.50 


0.43 


3.80 


3.33 


0.12 


12 


29 + _ = 30 


0.50 


0.57 


3.30 


3.47 


0,12 


13 


22 + 8 = 


0.50 


0.67 


4.80 


4.46 


0.80 


14 


_ + 2 = 25 


- 0.67 


0.53 


4.90 


4.21 


0.45 


15 


21 + 8 = 


0.67 


0.45 


3.70 


3.96 


1.16 


16 


+8 =30 


0.67 


0*81 


4.30 


4.82 


0.76 


17 


23 + =28 


0.83 


0.40 


2.90 


3.27 


4.72 


18 


_ + 3 = 24 


0.83 


0.46 


3.30 


4.01 


3.32 


19 


+7 =28 


0.83 


0.59 


3 »oo 


4.26 


1.43 


It 


18,03 (19 items) 


1 


(items < 10) 


= 18.03 (19 


items) 


« 0.65 
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Table 5 



Predicted and observed proportions of errors and success- latency in 



fourth-grade addition, concept block 1, level 3 







Observed 


Predicted 


Observed 


Predicted 


p 


Rank 


Equations 


(1 “ Pi) 


(1 - Pi) 


Latency 


Latency 




1 


26 + =29 


0.02 


0.28 


3.40 


5.16 


7.96 


2 


22 + _ = 2^ 


0.02 


0.18 


2.90 


4.58 


4.27 


3 


24 + 0 = __ 


0.02 


0.04 ■ 


2.10 


2.41 


. 0.24 


k 


26 + 0 = _ 


0.02 


0.05 


2.30 


2.6 t 


0.45 


5 


36 + 0 = _ 


0.03 


0.16 


2.30 


3.99 


2.22 


6 


31 + 0 = _ 


0.03 


0.09 


2.00 


3*33 


0.86 


7 


37 + _ - 39 . 


0.03 


0.10 


2.80 


3.58 


1.00 


8 


40 + = 4o 


0.03 


0*05 


2,50 


2.51 


0.13 


9 


26 + 1 = _ 


0.04 


0.11 


3.40 


3.66 


1.02 


10 


37 + _ = ^0 


0.06 


0.20 


3.60 


4.65 


2.41 


11 


24 + _ = 27 


0.08 


0.23 


4.40 


4.89 


3.02 


12 


22 + 1 = _ 


0.08 


0.07 


2.70 ' 


3.13 


0.09 


13 


22 + __ = 25 


0.08 


0.19 


3.30 


4.63 


1.84 


Ik 


35 + _ = 40 


O.li 


0.23 


3.00 


4.82 


i.40 


15 


33 = 39 


0.11 


0.13 


4.00 


3.92 


0.04 


l6 


27 + 0 = _ 


0.13 


0.06 


3.10 


2.81 


2.01 


17 


23 + _ = 27 


0.13 


0.22 


3.70 


4.81 


1.27 


18 


21 + __ = 30 


0.17 


0.37 


4.80 


5.72_ 


4.21 


19 


31 + _ = 34 


0.17 


0.08 


4.lf0 


3.43 


1.61 


20 


_ + 1 = 37 


0.17 


0.45 


4.50 


5.92 


5.9^^ 


21 


33 + 6 = _ 


0.28 


0.15 


4.30 


4.02 


2.56 


22 


_+2 = 23- 


0.29 


0.12 


5.70 


3.98 


6.45 


23 


_ + 2 = 26 


0.29 


0.17 


5.30 


4.38 . 


2.68 


2 k 


23 + 7 = _ 


C.33 


0.18 


6.6c 


4.48 


3.^9 


25 


_ + 3 - 35 


0.33 


0.36 


5.50 


5.48 


0.05 


26 


32 + 4 = _ ■ 


0.33 


0.15 


4.30 . 


4.06 


4.86 


27 


31 3S 


0.33 


C.I3 


4.90 


3.96 


6.5^^ 


28 


+ 4 = 38 


0.39 


0.43 


5.70 


5.79 


n.i2 










Table 5 (continued) 







Observed 


Predicted 


Observed 


Predicted 


p 




aquations 


(1 - Pi) 


(1 - Pi) 


Latency 


Latency 


yr 


29 


32 + _ - 39 




0 . 1.4 


5.20 


4.00 


14.57 


30 


__ + 1 == 32 


0 A 4 


0,31 


6.30 


5.26 


1.51 


31 


26 + 3 = _ 


0.46 


O.iJL 

* 


4.50 


3.75 


27.76 


32 


_ + 5 = 30 


0 . 54 - 


0.35 


5.70 


5.59 


3.77 


33 


__ + 6 = 39 


0,56 


0.42 


6.10 


5.75 


1.31 


3 ^ 


! 

ON 

ii 

ro 

00 


0.58 


0.16 


5.90 


4.30 


31.77 


35 


_ + 9 = 30 


0.63 


0.29 


5.40 


5.24 


13.19 


36 


_ + 6 = 27 


0.67 


0.14 


5.50 


4.17 


52.85 


37 


_ + 5 = 37 


0.67 


0.38 


7.20 


5.58 


6.20 


38 


+ 4 = ifO 


0.83 


0.67 


8.10 


6.99 


2.27 


= 


233»91 (38 items; 


) 


(items < 10) 


= 83.78 (33 items) 


= 1.29 



Tuble 6 

Predicted and observed proportions of errors and success-latency in 
fourth-grade addition, concept block 1, level k 



Bank 


Equations 


Observed 

(1- P^) 


Predicted 

(1 - Pi) 


Observed 

Latency 


Predicted 

Latency 




1 


0 + 29 = 0 -t- 


0.01 


0.03 


2.40 


2.23 


0.40 


2 


1 + 38 = +0 


0.05 


0.13 


3.20 


3.72 


0,69 




1 +■ 27 C 5 0 +• 


0.06 


0.16 


2.60 


3 e 97 


2.84 


k 


2 + 36 s 0 + 


0.09 


0.13 


3.80 


3.74 


0.16 


5 


0 34 = 0 


0.09 


0.03 


3.40 


3.12 


1.45 


6 


3 + 26 = C + 


0.14 


0.16 


3.70 


3-95 


0.08 


7 


1 + 25 = _ -f- 0 


0.17 


0.16 


2.90 


4.0a 


0.00 


8 


4 •{• 27 = 2 + 


0.27 


0.68 


7.30 


6.33 


8.30 


9 


33 3 = _ + 29 


0.27 


0.79 


5.70 


6=86 


1731 


10 


, 12 + 10 == +0 


0.28 


0.18 


4.50 


4.11 


2.53 


11 


10 + 16 = 6 + 


0.39 


0 . 4 l 


3*90 


5.00 


0.04 


12 


27 2 = 10 + 


0.39 


0 . 6 ^ 


0 

CM 

• 

ITS 


5 .78 


9.75 


13 


7 + 18 = 0 + _ 


0.39 


0.38 


4.50 


5-19 


0.01 


Ih 


9 + 28 = 7 + _ 


0.46 


0.77 


6.50 


6.53 


5.91 


15 


10 + 29 = 8 + _ 


0.46 


0.53 


5.00 


5.41 


0.26 


l 6 


11 + 12 = +1 


0.53 


0.56 


5.80 


5.»r 


0.16 


IT 


24 + 3 = 5 + _ 


0.58 


0.46 


5.00 


5.35 


2.37 


18 


17 + 5 = _ + 11 


0.58 


0.40 


6.20 


5 ..pa . 


5.04 


•19 


9+14 = +2 


0.58 


0.72 


7.50 


6.31 


3.11 


20 


9 + 18 = +5 


0.61 


0.76 


7.40 


6.63 


4.68 


21 


34 + 5 = 11 + _ 


0.64 


0.59 


5.90 


5.78 


0.11 


22 


22 + 7 « + l4 


0.72 


0.69 ' . 


7.60 


6.15 


0.19 


23 


7 + 22 = 6 + 


0.72 


0«53 


6.00 


5.50 


5.37 


2 h 


11 + 28 = 3 + 


0.73 


0.67 


6.00 


,5.98 


0.19 


25 


27+7 = +20 


0 oT 3 


0.66 


6.40 


6.03 


0.19 


26 


17 + 22 * 5 + _ 


Oc 73 


0.59 


7.00 


5.78 


0.90 


27 


35 + 3 == _ + 12 


0.73 


0.54 


tm mm m 




1.62 


28 


30 + 2 = +5 


0.73 


0.54 

s 


4.60 , 


5.74 


1.54 



mic 



Table 6 (continued) 



Rank 




Equations 


Observed 
(I . Pi) 


Predicted 
(1 - Pi) 


Observed 

Latency 


Predicted 

Latency 


x" 


29 


23 


+ 2 = + 8 


C.75 


0.83 


7.00 


6.88 


1,46 


30 


25 


+ = 11 + 


0.75 


0.61 


5.70 


5.91* 


2.90 


31 


19 


+ 8 s= + 6 


0 . 75 ’ 


0.78 


6.40 


6.70 


0.25 


32 


32 


+ 5 = +9 


0.91 


0.72 


— 


— 


1.91 


33 


29 


+ 7 = _ + 15 


0.91 


0,85 


6.10 


7.13 


0.26 


31^ 


22 


+ 12 =. 16 + 


O.Sl 


0.95 


8ao 


8.09 


0.39 


35 


12 


+ 22 = _ + 6 


0.9.1 


0.85 




— 


0.34 


36 


33 


+ 1 = 7 + 


0.9] 


0.64 


7.60 


6.19 


3.49 


37 


Ik 


+ 10 = 9 + 


0.92 


0.84 


6.30 


6.97 


1.43 


38 


29 


+ 3 = _ +. 17 


0.96 


0 s 93 


--- 


— 


0.13 



= 87.94 (38 items) X^(iten,s < lO) =70.43 (37 items) 



= 0.73 



fs/f 



- j P k^ nr 










Table 8 

Predicted and observed proportions of errors and success-latency in 




foxirth- grade additior^ concept block :j, level 2 







Observed 


Predicted 


Observed 


Predicted 




Rank 


ESquations 


(1 " Pi) 


( 1 .- Pi) 


Latency 


Latency 




1 


32 + _ - 33 


0.07 


0.15 


2.90 


3.08 


0.33 


2 


35 + _ = 37 


0.07 


0.16 


2.80 


3‘07 


0.39 


3 


37 + 0 = _ 


0.07 


0.12 


3.20 


. 3.23 


o.i 4 


i|- 


39 . + 0 =-• _ 


0.07 


0.13 


2.80 


3.35 


0.20 


5 


33 + _ = 36 


0.07 


0.16 


4,00 


3.18 


0.43 


6 


34 + _ - 35 


0.07 


0.15 


2.10 


3.03 


0,3^ 


7 


32 + _ = 32 


0.07 


0 .l 4 


2.70 


3.02 


0.29 


8 


47 ’ - 47 


0.07 


0.15 


1.80 


2.66 


0.36 


9 


4 l + 7 _ 


0 e 07 


U.26 


4.90 . 


4.44 


1.34 


10 ' 


43 = 43 


0.07 


0.15 


1.90 


2.76 


0.34 


11 


42 + _ = 42 


0.07 


0.15 . 


1.00 


2.78 


0.33 


12 


47 + 0 = __ 


0.07 


0.18 


3.20 


3.85 


0 . 55 - 


13 


45 + 0 = _ 


0.08 


0.16 


3.00 


3.72 


0.29 


14 


43 + = ^ . 


0.08 


0.16 


2.90 


■ 2.82 ' 


0.24 


15 


455 + = 46 


0.08 


0.16 


2.80 


2.77 


0.25 


l 6 


33 + _ = 34 


0.14 


0.15 


2.40 


3.05 


0.00 


17 


38 + 2 = __ 


0.14 


C.39 


4.90 


5.50 


1.82 


18 


39 + _ = 40 


0 .l 4 


0.28 


2.40 


4.04 - 


0.65 


19 


32 + 3 = _ 


0.14 


0 .l 8 


4.30 


3.98 


0.07 


20 


44 + 1 = _ 


0.14 


0.29 


2.50 


4.77 


0.71 


21 


47 + 3 - _ 


0.14 


0.90 


3.60 


6.03 


3.66 


22 


43 + = 46 ' 


0.14 


0.17 


• 2.30 


2.94 


0.04 


23 


_ + 1 = 47 


O.ll* 


0.49 


4.70 


6.02 


3.36 


24 


__ + 0 = 44 


0.17 


0.29 


2.90 


4.79 


0.42 


25 


45 + __ = 49 


0.17 


0.18 


3.50 


2.96 


0.01 


26 


4 l + ___ = 45 


0.17 


0.18 


3.90 


3.05 


0.00 


27 


_ +■ .0 = 47 


0.17 


0.32 


3.50 


4.97 


0.63 


28 


49 + 1 = _ 


0.17 


0.43 


3.80 


5.64 


1.71 













Table 8 (continued) 



Ranlc 


Equations 


Obsei’ved 
(1 - Pi) 


Predicted 
(1 - Pi) 


Observed 

Latency 


Predicted 

Latency 




29 


_ 0 = 34 


0.29 


0.20 


2.00 


4.17 


0=37 


30 


31 + 7 = _ 


0,29 


0.18 


6.60 


3.82 . 


o.p 


31 


35 + _ = 39 


0,29 


. 0.17 


3.00 


3.19 


0.66 


32 


4 l + 4 = 


0.29 


0.26 


3.20 


4.51 


0.02 


33 


+ 2 = 50 


0.29 


0.70 


6.40 


7.24 


5.56 


3 ^ 


43 + 6 = _ . 


0.29 


0.28 


5.60 


4.59 


0.00 


35 


^3 + 2 - _ 


0.33 


0.28 


6 . 4 o 


4.68 


0.09 


36 


_ + 1 = 45 


0.33 


0.46 


7.50 


5.89 


0 . 4 l 


37 


44 + 5 = _ 


0.33 


0.29 


7.40 


4.67 


0,05 


38 


42 + 3 = _ . 


0.33 


0.27 


5.80 


4.60 


0.13 


39 


42 + 0 _ 


0.33 


0 .l 4 


4.60 


3.54 


1.74 


40 


31 + = 40 


0.43 


0.36 


5.00 


4.72 


0.16 


' 4 l 


— + 2 ^ 36 


0.43 


0.35 


6.60 


5.25 


0.22 


42 


45 + 4 = _ 


0.43 


0.30 


3.40 


4.76 


0.55 


^3 


43 + _ == ^7 


0.43 


0.18 


4.06 


3.00 


3.07 


44 


43 + 7 = _ 


0.43 


0.46 


4.40 


5.69 


0.03 


45 


42 + __ = 45 


0.50 


0.17 


4„80 


2.96 


4.70 


46 


4 l + 9 « _ 


0.57 


0.44 


3.90 


5.52 


0.51 


4 ? 


_ + 3 = 50 


0.57 


0.69 


5.70 


7.16 


0.43 


48 


+ 3 = 49 


0.57 


0.49 


6.40 


5.97 


0.18 




0 

if\ 

II 

1 

+ 


' 0.67 


0.34 


6.10 


.4.23 


2.88 


50 


32 Hf- _ = 33 


0.71 


0.15 


2.60 


3.08 


17.77 


51 


32 +7 = _ 


0.71 


0.19 


3.80 


3.89 


12.94 


52 


_ + 3 = 37 


0.71 


0.35 


9.40 


5.23 


4.18 


53 


_ + 4 - 46 


0.71 


0=44 


7.40 


5.70 


2.08 


54 


42 + 7 = „ 


0.83 


0.27 


6.00 


4.50 


9.46 


55 


• _ •+ 8 = 50 


0.83 


0.64 


7.30 


. 6.73 


1.00 


56 


__ + 6 = 48 


0.83 


0.45 


9.40 


5.65 


3.64 


57 


44 46= ; 


0.92 


0,47 


— . 




4.80 


y ? 


= 97.02 (57 items) (items < lO) = 


66.31 (55 items) 


= 2.29 



o 



; 







tS. 



i: 






'i 








Table 9 

Predicted and observed proportions of errors and success-latency in 



fourth-grade additon, concept block 3 > level 3 







UD served 


Predicted 


^ 

vueervau 


..Ik .S 


.z 


Rank 


Equations 


(1 - Pi) 


(1 - Pi> 


Latency 


Latency 


nr 


1 


+0 « 69 


0.01 


0.05 


3.30 


3.78 


1.39 


2 


65 + 0 • 


0,01 


0.02 


2.20 


2.29 


0.^ 


3 


_ + 0 » 62 


0.01 


0.07 


3.60 


3 «T 2 


1.9a 


4 


66 + *s 66 


0,01 


0,02 


1.40 . 


1.31 


0.13 


3 


^ 0 a 63 


0.01 


0.06 


3.10 


3.73 


1.83 


6 


^ S5S 65 


0.01 


0.03 


1.90 


2.15 


o.d» 


rif 

( 


_ + 0 « 70 


0.01 


0.05 


2.90 


3 o 78 . 


1.32 


6 


39 + -39 


0.02 


0.10 


2.00 


2.69 


3.58 


9 


44 + ^ * 47 


0.02 


o.n 


2.90 


3.19 


1.95 


10 


51 + 4 « _ 


0.02 


0.07 


3 . 4 o 


3.43 


0.78 


11 


52 + _ • 52 


0.02 


0.05 


1.60 


2.03 


0.28 


12 


61 4 « 


0.02 


0.05 


3.30 


3.51 


0.70 


13 


•f 1 c 


0.02 


0.15 


4.30 


5.12 


5.13 


U 


37 + = 37 


0.04 


0.11 


2.10 


. 2.79 


2.68 


15 


34 + 1 « _ 


0.04 


o.i4 


2.60 


3.44 


4.08 


16 


+ 0 as 39 


0.04 


0.13 


3.60 


3.54 


3.>*7 


17 


49 + _ * 49 


0.04 


0.06 


2.20 


2.18 


0.02 


18 


-F 0 a 48 


0.04 


0.10 


3.90 


3 . 6 X_ 


0.94 


19 


48 + « 9 C 


U .04 


0.20 


3 . 2 Q 


4 .43 ' 


3.9T 


20 


54 + _ • 57 


0.05 


0.06 


4.00 


2.68 


0.09 


21 


54 + a 5® 


0.05 


0.06 


3.00 


2.68 


0.37 


22 


56 + 1 as 


0.05 


0.07 


3.10 


3. >2 


0.26 


23 


51 + _ = 51 


0 . 0 > 


0.05 


1.50 


2.08 . 


O.Cl 


24 


59 + _ « 59 


0.05 


0.03 


1.80 


1.67 


0.15 


25 


•.». + 0 60 


0.05 


0 .C 7 


4.00 


3.70 


0,20 


26 


5.1 > 5 - _ 


0.05 


D.Or^ 


3.50 


3^38 


0.18 


27 


61 + ^ « 65 


0.05 


0.04 


2.50 


3.33 


0.13 


28 


67 + 3 « 


.05 


0.11 


3*40 


5.06 


1.75 














■ ■' 



Table 9 (continued) 



Rank 


Equations 


Observed 
(1 - Pi) 


Predicted 

(1 - p^) 


Observed 

latency 


Predicted 
liatency . 




29 


_ + 0 = 35 


0.06 


0 -l 4 


3.50 


3.50 


2.76 


30 


31 + 1 = _ 


0.06 


0.15 


2.40 


3.42 


3.16 


31 


6 l 4 7 = _ 


0.07 


• 0.04 


3.20 


3.35 


0.78 


32 


41 + 2 = _ 


0.08 


0.11 


2.60 


3.45 


0.64 


33 


_ 4 1 = 33 


0.08 


0.30 


4.10 


4.88 


12.31 


3 ^ 


31 + _. = 36 


0.08 


0.20 


3 el 0 . 


3.87 


4.78 


35 


42 + = 4 !*. 


0.08 


0.12 


2.30 


3. as 


0.41 


36 


43 + 4 = 


0.08 


0.14 


4.90 


4.09 


0.70 


37 


52 + 2 = _ 


0.09 


0.08 


2.90 


3.54 


0.05 


38 


^ + 1 =67 


0.10 


0.13 


4.10 


5.16 


0.40 


39 


^ + 3 = 


0.10 


O.C^ 


3.50 


3.59 


1.96 


40 


46 + 0 = _ 


0.11 


0.04 


2„90 


a . 14 


6.61 


kl 


31+2 a 


0.12 


0.14 


2.60 


3.37 


0.20 


42 


43 + 45 


0.12 


0.12 


3.30 


3.23 


0.01 


43 


48 + _ = 49 


0.12 


0.09 


2.70 


a.s 7 


0.80 


41 ^ 


63 + 7 = _ 


0.12 


0 . 3.0 


4.50 


4.83 


0.17 


45 


_ + 1 a 69 


0.12 


0.12 


4.60 


5.17 


0,00 


46 


_ + 0 a 49 


0.14 


O.IO 


3 o 6 o 


3.68 


1.29 


47 , 


54 + 4 a ___ 


0.14 


0.07 


3 c 40 


3.45 


1.79 


48 


51 + _ = 57 


0.14 


0.07 


3.30 


a.85 


1.81 


49 


36 + 3 * _ 


0.14 


0.12 


3.20 


■3.-36 


0.23 


50 


32 + _ = 36 


0 ,l 4 


0.20 

« 


3.50 


3.81 


1,12 


✓ -- 


43 + 3 = _ 


0.15 


0.09 


3.40 


3.41 


2.50 


52 


31 + _ = 37 


0.16 


0.20 


3.20 


3.87 


0.49 


53 


+ a 48 


0.16 


0.11 


4.40 


3.31 


0.67 


5>4 


U3 + =48 


0.16 


0.11 


4.30 


3.25 


0.77 


55 


45 -«■ 3 = 


0.16 


0.09 


3.90 


3-43 


1*51 


56 


46 4 * s= 46 


O0I7 


0.07 


2.20 


2.33 


10.76 


57 


^ + 4 a 66 


O.IT 


0.12 


5.00 


4.97 


0.81 


58 


_ + 2 a 40 


' 0.16 


0.47 


5.60 


6.34 


1742 


59 


4 l + 0 a 


0.15 


0.05 


2.80 


a. 10: 


24.98 








Table 9 (continued) 









Observed 


Predicted 


Observed 


Predicted 




Equations 


(1 - Pi) 


(1 - ?i) 


Latency 


Latency 




Cl 


- 1 * 




^ 1 Q 


r\ A/iC 






c An 




V 








^•-rw 




y 


57 


+ 


= 60 


0.18 


0.08 


4.40 


3.25 


3.27 


5U 


+ 


_ = 6o 


0.18 


.0.09 


4.70 


3.43 


2.62 




+ 


_ = 38 


0.20 


0.18 


2.90 


3.70 


O0I3 




+ 


1 = 


0.20 


Ocll 


2.70 


3.50 


4^75 


kl 


-1. 

« 


^ _ 

\J ss 

«raMMT» 


0.20 


0.08 


3.60 


3.25 


I0c73 


kl 


+ 


_ = 48 


0.20 


o.u 


4.00 


3.37 


1.94 






0 =43 


0.21 


0.11 


3.30 


3.57 


6.58 


k 6 


+ 


1 = 


0.23 


0.10 


2.40 


3.54 


12.52 






2 =55 . 


0.23 


0.18 


6.20 


5.00 


0.40 


42 


+ 


4 =_ 


0*24 


0.09 


4.40 


3.36 


6.51 


45 


+ 


5 


0.24 


0.18 


3.80 


4.78 


1.43 


4? 


+ 


_ = 50 


0.24 


0.14 


2.70 


3 o 76 


6.14 




+ 


2 = 38 


0.26 


0.27 


5.70 


4.87 


0.03 


44 


•h 


_ = 44 


0.26 


0.07 


1.50 


2.43, 


32.14 




f 


3 = 64 


0.27 


0.14 


5.30 


5.01 


6.19 


52 


+ 


8 =_ 


0.27 


0.13 


5.60 


• 4.69 


3.86 




+ 


5 = 39 


0.28 


0.24 

t 


4.90 


4.70 


0.30 




+ 


= 48 


0.28 


0.20 


6.00 


4.83 


1.02 




+ 


2 = 43 


0 o 2 B 


0.24 


6.50 


4.91 


0.26 




+ 


5 =50 


0.28 


0.37 


6.00 


6,24 _ 


0.90 


4 l 


+ 


8 =_ 


0.28 


0.08 


4.60 


3.14 


14.97 


42 


+ 


_ = 4 ? 


0.30 


0.11 


3.20 


3.30 


24.15 . 




+ 


7 = 38 


0.31 


0.24 


4.40 


4.57 


1.70 




+ 


3 - 69 


6.32 


0.12 


5.50 


5.05 


15.62 




+ 


2 = 47 


0.32 


0.21 


7.10 


4.94 • 


1.64 




+ 


4 = 50 


0.33 


0.38 


6.30 


6.30 


0c60 




+ 


5 = 46 


0.33 


0.20 


5.10 


4.75 


6.70 




• 

»<• , 

« 


2 = 60 


0.36 


0.32 


5.60 


6.50 


0.17 




* 


9 «50 


0.38 


0.34 


5.40 


6.00 


0.38 




+ 


6 =38 


0.39 


C .24 


6.00 


4.63 


6.24 



Table 9 { continued) 







Observed 


Predicted 


Observed 


Predicted 




Rank 


Equations 


(1 - s.) 

i 


U - P^) 


Latenqr 


Latency 




91 


_ + 3 =45 


C.^A 


0.22 


5-50 


4.86 


7.04 


92 


+ 3 = ^0 


0.47 


0.46 


6.00 


6.28 


0.02 


93 


_ + 5 = ^9 


0.52 • 


0.19 


5.70 


4.78 


45 .44 


9*^ 


+ If » 45 


0.52 


0.21 


6.90 


4,80 


33«8 i 


93 


+ 4 s= 60 


Oc55 


0.31 


7.00 


6.38 


5.7*^ 



= 391.87 (95 items) X^(items < IC) = 156.75 {83 items) = 0.62 
















Table 10 




Predicted and observed proportions of errors and success-latency in 
fourth-grade addition, concept block 3> level 







Observed 


Predicted 


Observed 


Predicted 




Rank 


Equations 


(1 - Pi) 


(1 - Pi) 


Latency 


Latency 


y ? 


■ 1 


2 + ^4-6 = 0 + 


0.02 


0.17 


3.70 


. 4.14 


7.23 


2 


7 + 50 == 0 + _ 


0.02 


0.12 


3.20 


3.80 


3.68 


3 


0 -f 46 - 0 + 


0.06 


0.05 


2.50 


2.70 


0.28 


k 


1 + 54 = _ + 0 


0.07 


0.17 


3.00 


4.24 


3.24 


5 


56 + 1 = 5^ + _ 


0.11 


0.45 


3.50 


5.70 


20.61 


6 


69 + 0 = + 9 


0.13 


0.12 


3.20 


3.98 


0.12 


7 


47 0 = 45 + 


0.15 


0.16 


2.90 


4.12 


0.12 


8 


48 + 21 = 21 + _ 


0.17 


,0.66 


5»90 


6.77 


24.63 


9 


32 + 20 = 0 + _ 


0.21 


0.17 


5.20 


4.20 


0.44 


10 


.4 + 50 = 3 + 


0.23 


0.25 


4.50 


4.76. 


0.16 


11 


63 + 4 = 7 + __ 


0.26 


0.37 


5.30 


5.44 


1.13 


12 


57 + 0 + 7 


0.27 


0.32 


3.3c 


3.80 


5.36 


13 


7 + 40 = 6 + _ 


0.29 


0.35 


3.90 


5.19 


0.79 


14 


4 + 42 = 1 + 


0.29 


0.33 


4.90 


5.07 


0.37 


15 


7 + 5^ = 1 + __ 


0.30 


0.57 


5.60 


6.23 


6.85 


l6 


52 + 10 =r +11 


0.30 


0.50 


5.30 


5.96 


3.50 


17 


45 + 12 = _ + 12 


0.36 


0.61 


4.10 


6.4o 


5.82 


13 


8 -}■ 51 ” 5 + _ 


0.36 


0.36 


4.60 


. .5.3!^ 


0.00 


19 


23 30 - _ + 2 


0.36 


0.34 


6.20 


5.19 


0.04 


20 


5 ^ 52 = 3 + _ 


0.36 


0.35 


4.20 


5.27 


0.01 


21 


22 4 20 == _ + 10 


0.40 


0.36 


5.40 


5.20 


0.22 


22 


9 + 42 * 1 + 


0.41 


0.56 . 


5.20 


6.08 


2.09 


23 


30 + 26 = 10 + 


0.41 


0.38 


5.20 


5.40- 


0.17 


24 


4 + 64 := _ + 2 


0.44 


0.36 


.5.30 


5.42 


0.54 


25 


12 + 56 = +0 

«<SHM 


0.44 


0.26 


6.30 


4.90 


3.84 


26- 


67 + 1 = _ + 30. 


0.44 


0.36 


7. 00’ 


5^39 


0.61 


27 


7 + 37 = _ + 2 


6.44 


0.56 


7.10 


5.99 


2.80 


28 


7 + 45 = 3 + _ 


0.46 


0.77 


5.10 


7.07 


12.31 



V 









Table 10 (continued) 



1 




, Observed 


Predicted 


Observed 


Predicted 


2 


Rank 


Equations 


(1 - Pi) 


(1 - Pi) 


Latency 


Latency 


X 


29 


11 + 40 = 3 + 


0.46 


0.57 


5.70 


6c12 


1.20 


30 


4 ? + 8 = 1 + __ 


0.46 


0.57 


6.80 


6.14 


2.24 


31 


50 + 19 = 45 + _ 


0.52 * 


0.55 


6.60 


6.26 


0.05 


32 


36 + 33 = 13 + _ 


0.52 


0.63 


6.90 


6.60 


1.20 


35 


24 + 22 = _ 20 


0.5^ 


0.52 


7.40 


5.93 


0.08 


3 ^ 


32 + 22 = 20 + _- 


0.55 


0.53 


5.40 


6.05 


0.02 


35 


60 + 5 = 21 + _ 


0.57 


0.37 


6.90 


5.43 


3.77 


36 


5 + 36 = ^ 


0.58 


0.67 


6.60 


6.46 


1.67 


37 


29 + 30 = 13 + _ 


0.59 


0.51 


5.60 


5.98 


0.60 


38 


10 + 41 := _ + 7 


0.59 


0.59 


5.50 


6.20 


0.00 


39 


22 + 31 __ + 20 


0.59 


0.53 


6.90 


6.04 


0.64 


ko 


51 + 13 = _ + ^1 


C.61 


0.63 


' 6.80 


6.52 


0.03 


hi 


33 + 35 = _ + 13 


0.61 


0.63 


6.50 


5.68 


0.05 


k 2 


48 w 10 = 23 


0.64 


0.49 


5»70 


5-90 


1.77 


^3 


15 + 40 = _ + 7 


0.64 


0.59 


6.10 • 


6.26 • 


0.18 


kk 


54 + 4 = _ + 27 


0.64 


0.47 


6.10 


5.78 


4.94 


^5 


5 + 56 = _ + 3 


0.65 


0.58 


7.26 


6.27 


0.46 


) 


50 + 2 = + 13 


0.68 


0.45 


5.30 


5.65 


4.66 


Hi 


23 ;- 29 = 8 4 


0.68 


0.85 


4.60 


7.65 


5.07 


hQ 


42 + 2 = _ + 9 


0.69 


0.56 


7.60 


5.99 


3.30 


^9 


42 4 15 = 32 '-• _ 


0.71 


0.63 


6.70 


-- ' 6.46 


1.16 


50 


29 + 26 = _ + 5 


0.71 


0.69 


6.60 


6.69 


0.05 


51 


8 + 36 = 3 + _ 


0.71 


0.56 


6.40 


6.02 


4.16 


52 


51 + 8 = + 35 . 


0.73 


.0.49 


6.90 


5.88 


10 . i 5 


.53 


37 + 15 = 31 + _ 


0.^3 


6.80 


7.10 


7^32 


1.66 


5 ^^ 


19 + 26 = _ + 3 


0.73 


0.67 


7.60 


6.50 


0.73 


55 


23 + 25 := + 14 


0.75 


0.61 


7.60 


6.31 


. 3 . 85 


56 


17 + 37 = 13 + _ 


0.75 


0.80 


6.10 


7.31 


0.69 


57 


54 + 4 - 23 + 


0.77 


0.47 


6.80 


5.78 


8.18 


58 


29 + 39 = _ + 21 


0.78 


0.83 


8.10 


?c69 


A oA 
^ • ^0 






I 




miattm 



Table 10 (continued) 



Rank 


Equations 


Observed 
(1 - Pi) 


*59 


21 + 47 = _ + 16 


0.78 


60 


39 + 29 = 13 + _ 


0.78 


61 


25 + 39 = _ + 13 


0.78 . 


62 


17 + 24 14 + __ 


0.81 


63 


24 + 27 = _ + 16 


0.82 


64 


.36 + 17 = 8 + ___ 


0.82 


65 


16 + 38 = _ + 13 


C.82 


66 


25 + 19 = _ + 17 


0.83 


67 


29 + 12 = __ + 5 


0.83 


68 


28 + 25 = 13 + _ 


0.84 


69 


52 + 1 =. 24 + _ 


0.84 


70 


28 4- 18 = __ + 21 


0.85 


71 ' 


31 + 24 = _ + 19 


0.89 . 


72 


19 + 22 = + 4 


0.90 


73 


35 + 20 = 28 + _ 


0.91 


74 


53 + 1 = + 18 


C.91 


75 


35 + 9 = 23 _ 


0.92 


76 


11 + 51 = 9 + _ 


0.96 


2 

X = 


225.08 (76 items) 


X^( items 



Predicted 


Observed 


Predicted 


p 


(1 - Pi) 


Latency 


Latency 




0.64 


7.40 


6.65 


1.97 


0.81 


7.40 


7.52 


0.12 


0.81 


7.80 


7.46 


0.09 


0.91 


7.00 


8.08 


4.81 


0.91 


8.50 


8.27 


2.45 


0.85 


8.00 


7.66 


0.21 


0.80 


7.30 


7.31 


0.09 


0.91 


6.90 • 


8.18 


3.61 


0.84 


7.60' 


7.42 


0.01 


0.30 . 


7.30 


7.30 


0,47 


0.67 


8.5c 


6.58 


5.68 


0.81 


7.80 


7.30 


0.67 


0.82 


6.60 


7.45 


1.41 


0.84 


7.90 


7.40 


1.25 


0,74 


7^80 


,7.01 


• 3.23 


U.67 


7.60 


6.59 


11.10 


0.69 


9.00 • 


6.61 


11.37 


0.71 


7.10 


6.88 


6.79 



10) ^ 13^.90 (70 items) 



= 1.04 



Table 11 

Predicted and observed proportions of errors and success-latency in 
fifth-grade addition, concept block 3> levels 3 and 



Bank 


Equations 


Observed 
(1 - Pi) 


Predicted 
(1 - Pi) 


Observed 

Latency 


Predicted 

latency 




1 


•f 0 « 34 


0.14 


0.07 


2.90 


. 1.29 


0.49 


2 


32 + _ = 33 


0.l4 


0.13 


1.60 


2.67 


0.01 


3 


35 + _ = 37 


0.14 


0.18 


1.80 


3.10 


. 0.05 


4 


31 + « 40 


0.l4 


C.36 


3.70 


4.32 


1.13 


5 


37 + 0 =r _ 


0.14 


0.07 


1.50 


' 1.09 


0.47 


6 


39 c 40 


0.14 


0.25 


0.60 


3.77 


0.34 


7 


39 + 0 B 


0.14 


0.08 


2.00 


1.27 


0.29 


8 


32 + 3 = _ 


0.14 


0.11 


2.50 


2.05 


0.06 


9 


32 + « 38 


0.14 


0.23 


2.60 


3.47 


0.28 


10 


33 + _ = 36 


0.14 


0.18 


1.20 


3.08 . 


.0.05 


3,1 


34 + _ = 35 


0.14 


0.15 


1.40 


2.85 


0.00 


12 


35 + _ = 39 


0.14 


0.23 


l.4o 


3.42 


0.24 


13 


32 + _ e 32 


044 


0.11 


2.20 


2.51 


0.05 


14 


7 + 29 = 34 + _ 


0.14 . 


0.31 


4.70 


4.74 


1.75 


15 


32 + 6 « +36 


0.14 


0.27 


2.60 


4.13 


1.17 


16 


34+5 = 35 -J* 


0.14 


0.32 


2.40 


4.35 


1.93 


17 


33 + 4 s= ^ 0 


0.14 


0.16 


3.90 


2.96 


0.03 


18 


33 + _ = 34 


' 0.17 


0.14 


0.90 


2.76 


0.04 


19 


38 + 2 = _ 


0.17 


0.22 


3.30 


3.37 


0.09 


20 


32 + 7 « _ 


0.17 


0.18 


3.90 


2.69 


0.01 


21 


31 + 7 = _ 


0.17 


0.17 


3.20 


2.60 


0.00 


22 


_ + 3 = 37 


0.17 


0.16 , 


5.10 


2."'0 


0.01 


23 


31 + IT a 44 + 


0.20 


0.54 


4.90 


5.63 


7.16 


24 


15 + 22 a 4* 34 


0.21 


0.32 


5.00 


4.57 


0.68 


25 


5 + 4l ss 40 + 


0.27 


0.4l 


2.80 


4.58 


1.22 


26 ■ 


2 + 46 a _ + 43 


0.27 


0.51 


3.30 


5.49 


3*59 


27 


4 + 35 «= 4 + 


0.29 


0.32 


3.3c 


4.35 


C.06 


28 


1 + 37 « 24 + _ 


0.29 


0.31 

\ 


7.20 


4.52 


0.03 



. 
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Table 11 


( continued) 












Observed 


Predicted 


Observed Predicted 


P i 


Rank 


Equations 


(1 - Pj^) ’ 


(1 - Pj) 


Latency 


Latency 


X 1 : 

1 i 


on 


> 0 _ 




n K 


1. /Tr\ 


r\ cri. 


T ."V\ 1 L 




IT C. — JV/ 


VOO 




H- #UV 


C 


1 


30 


21 + 25 43 + 


0.33 


Oc54 


2 o 90 


5.85 


2.71 i 


31 


18 + 19 = 10 + 


0.36. 


0.54 


6.40 


. 5.99 


1.81 


32 


7 + 30 = _ + 28 


0.36 


0.46 


. 5.70 


5.^^5 


0.63 J 


. 33 


21 + 15 = 15 + 


0.36 


0.54 


3.10 


5.78 


1.91 - 


3h 


24 + 12 = + 4 


0.43 


0.31 


7.10 




0.88 1 


35 


22 + 16 s + 10 


0.43 


0.44 


6.20 


5 - 111 . 


0.01 J 


36 


20 + 19 5 + _ 


0.43 


0,38 


4.20 


4.89 




37 


26 + 11 = 26 + _ 


0.43 


0.50 


3.80 


5.59 


0.25 'm 


38 


11 + 26 = + 7 


0.43 


0.38 


5.60 


4 .^ 


0.17 


39 


28 + 21 =. 11 + 


0.47 


0.72 


7.'^0 


6 . 6 f 


4.77 'm 


ho 


11 + 28 = .+ 3 


0.47 


0.35 


5.30 


4.75 


0.87 :^1 


hi 


3 + 36 = + 14 


0.50 


0.35 


7.70 


4.75 


1.35 'll 


h2 


47 + 2 = +13 


0.53 


0.53 


5.80 


5.58 


0 

• 

0 

0 


^3 


26 + 23 - 6 + 


0,53 


0.60 


5 cl 0 


5.85 


• i 


hk 


9 + 29 = 12 + __ 


0-57 


0.54 


6.00 


6.01 


0.05 J 


h5 


20 + 28 « +18 


0.60 


0.75 


6.70 


6.59 


1.78 1 


h6 


10 + 39 = 7 + _ 


0.67 


0.56 


6.20 


5.45 


0.75 


h'j 


7 + 41 = 13 + 


0.67 


0.59 


7.00 


5.83 


^•35 :1 


48 


23 + 23 13 + _ 


0.67 


0.70 


5.30 


6.53 


0.07 m 


49 


0 + 48 = +12 


0.67 


0.42 


6.70 


4.88 


3.69 j|l 


50 


0 + 47 = 12 + 


0.67 


0.40 


5.70 


4,79 


4.36 1 


51 


39 t 7 ^ 12 + _ 


0c67 


0,66 


7.30 


6.59 


0.00 3 


52 


14 + 24 = +16 


0.71 


0.56 


6.80 


5.89 


1 


53 


39 + 7 = 9 + _ 


0.73 


0^76 


8.30 


7.52 


o.g4 s 


54 


27 + 11 = 25 + _ 


0.79 


0.50 


7.70 


5.61 


4.60 1 


55 


12 + 34 = _ *?• 28 


0.93 


0.77 


9.40 


7.40 


2.18 m 


56 


19 + 27 = 38 + _ 


0.93 


0..72 


6.60 


7.13 


3. .27 9 


57 


34 r 13 = 29 + _ 


0.97 


0.80 






2.66 8 


= 


64.00 (57 items) 


X^(i terns 


< 10) « 64 . 


00 (57 items) 




2.32 m 

iBm 




















Some of the values obtained , as for example in Table If -s are 

extreniwiy high and ■would usually be an indication of a poor fit, but a 

closer look at the coraponp^^ts of the shows that 5 of* "tbe 3^ problems 

-.2 

in this analysis contribute more than two thirds of the jtal X . In 

2 

the particular case cited, the large reduction in X still does not 

% 

make the value of X^ such that the model would not normally be rejected. 
VJhen we do reduce the values shown in Tables 4-11 by removing the 
few extreme components whose individual contributions are equal to or 

2 

greater than 10, we find that in four of the eight cases we obtain a X 
value whose probability P is such that .1 < P < .9 under the null 
hypothesis. Sine*' calculation of the regression coefficients included 
the extreme problems, a recalculation of the regression coefficients 
omitting the few extreme problems from the data would yield better fits 
of the model to data than those obtained c 

The overall X ' s as well as the reduced X * s are shown at the 
bottom' of Tables 4-11. Perusal of these tables with particular attention 
paid to the items for which predictions are unsatisfactory, and thus the 
X^ contribution high, suggests immediately further analysis that incor- 
porates variables designed to handle special algorithms. ■ Moreover, in 
those cases for ■which ? < .1, it should be noted that the actual pre- 
dictions are mostly fairly good. Our viewpoint on this mattex' is that 
we hardly expected to fit the data exactly with such a small number of 
variables. 

Figure 3 presents graph of the predicted euid observed proportions 

Insert Figure 3 about hers 



29 




of errors as a function of observed rank order of observed difficulty. 

The data for these curves were drawn from foxirth-grade addition , block 1, 
levels 2, 3 and An inspection of the two curves shows a relatively 
good fit for the regression model, even in the heterogeneous case of 
problems dravm from different drills , levels and correspondingly 
different groups of children. 

There are several queilitative observations about the data of Tables 
4-11 we want to make at this point. In the first place, the lirst 
problem presented on ec^h day was deleted from all analyses when the 
results showed a short but significant warm-up effect. This rendered the 
initial, problem more difficult independent of structural variables. 

Happily it was not necessary to include order of presentation as a variable 
since there was no significant warm-up effect beyond the first problem of 
a drill. The sequential effects, if any, of errors oh the immediately . 
following problem have not yet been analyzed ^ystematically, but agaxn 
this does not appear to be a. very strong effect. Although we intend to go 
into this question in more detail on a subsequent occasion, the assumption 
of statistical independence of problem-items seems to be correct to a first 
approximation . 

Tables 4, 5, 8, and 9 report data on problems of the general form 

•* 

m + n ^ p, where any of m, n <.r p may be two-digit numbers. What is 
striking is that the hardest problems are to a very large extent of the 
form = P' ^ problems of Table 4 are of this kind 

(the problem being ranked in order of difficiaty from easiest to hardest), 
the last 7 of Table 5> “E^nd the last 13 of Table "^^he effect, is not as 
noticeable in Table b, although over half of the last 11 problems are of 
this fc m. Moreover, if we look at the easiest problems in these same 
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tables, the form + n = p is very much excluded. With the exception 

of + 0 s= p, it does not occur in the easy half of i'able ^4-, in Table 

5 the form does not appear among the first 19 least difficult items, and 

in Table 8, not among the first 22. The evidence on this point is more 

% 

mixed in Table Q* All in all, these results suggest that the transfor- 
mation steps defined in the theoretical section might well be broken into 

separately weighted classes to differentiate _ + n = p from m + = p. 

In some preliminary efforts aimed, at refining and improving the predic^'^ve 
results reported here we have had some success with this distinction. 

Although the predictive results from Tables 6, 7j 10' and 11 are far 
from the best that a mature theory should be able to offer, we are not 
dissatisfied with them as a beginning because of the relative difficulty 
of intuitively rank ordering the es^ected error rate of problems of the 
form ab + cd = ef + gh. The three variables that we consider bring a 
surprising amount of order to what appears at first glance to be a quite 
complex set of problem- items . 

We turn now to the success-latency data for the same problems of 
fourth- and fifth-grade addition. The predicted and observed latencies 
are also given in Tables 4-11, with the predicted values depending on the 
appropriate regression coefficients of Table 3 * As is clear from Table 
3, the multiple correlations obtained for the fit of the predicted latencies 
are very conparable to those obtained for the predicted responses, and 
Indicate that the success-latency data are as regular in range of variation 
as the response data. 



V 
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In the analysis of latencies we have restricted ourselves to the 
success* latencies, i.e., the latencies of correct responses, because of 
their direct relevance for the analysis of the structure of the . algorithms 
students use. Although error-latencies also contain much useful infor- 
mation, they include latencies of r.andom guesses, false starts and other 
heterogeneous factors that are not easily disentangled, in a few ca^es 
latency data were garbled in transmission from the school to the conqputer, 
and in such cases we have simply entered a blank in both the predicted 
ai'id observed eo3umns. 

There are various ways of evaluating the overall fit of the latency 

2 

predictions reported in these tables. The sta. .Stic S , already mentioned 

is given at the bottom of each table. Although this statistic may be used 

to find a significance level for the fit of the structiarsl models, at this 

2 

stage of investigation it seems more useful to interpret S directly . 

in terms of the gueintitative closeness of the predictions. When the 

2 

structural variable:; f. . are not random variables ^ then S is a good 

10 • t 

p 

estimator of the variance J~ of the errors in the prediction of the 

models. Taking the algebraic sign into account, the expectation of these 

errors is nearly zero and the assumption that they are norm^ly distributed 

2 

with variance a is approximately satisfied also, and so we may evaluate 
the p-^edictions of each table by looking at the magnitude of S, the 
bulk of the errors being within one standard deviation of the observed 
values. The values of S for Tables i^-11 are .81, l.l4, .85> 1.08, 

1 . 51 , .79> 1*02 and 1 . 52 , respectively, which may be interpreted to mean 
that errors of prediction greater than 1 or 1.5 seconds do not 
occur very often. From inspection of the tables it may also be seen 




that the observed values have a range from about 3 seconds (Table 4) to 
more than 8.5 seconds (Table ll), and consequently, predictions of this 
accuracy are far from perfect, yet good enough to be practically useful. 

4 

another useful measure : 3 the average percentage .error of the 
predictions. If there are n items in a table, if o., is the mean 
observed success-lacency for item i', and p^^ is the predicted latency, 
then the average percentage error (A.E.) is defined by 



A.E 






This measure for Tables 4-11 has the values l6.¥-, 19.g^., 12,3^, 20 . 6 %, 
2o.7/^, lS.6%, 12,8% and 31*4^> respectively. 

The qualitative remarks made about proportion of errors pretty well 
apply to success-latencies, a-s would be expected because of the high 
positive correlation between the two variables. This -is particularly true 
of latencies for problems ot the form + n = p, as the reader may 
easily conclude from an inspection of Tables 4-11. 

Figure 4 presents a graph of the predicted and observed, success- 
late’iicies for the same problem- items for which response predictions are 

t 

shown in Figure 3* The predicted and observed latencies are plotted as 

Insert Figure 4 about here 

a function of the rank of observed latency, and consequently, the curve of 
observed latencies is monotonically increasing and smoother than the 
predicted curve, but the fit is qualitatively reasonably good. 
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Subtraction— grades four aM five . The three independent variables 
used in the linear regression analyses of subtraction %fere NSTEPS as 
described previously and the tvo i^gnitude variables ^ loagnitude of the 
difference (MAGDI?) and magnitude of the subtrahend (MAGSUB) . The values 
of MAGDIFF and MAGSUB are not affected by the problem format. For exan5>le, 

in all three problems , "^1 - l6 = 51 . ■ ~ 15 “ 1^ = 15 > MAGDIF 

has the value 15 and MAGSUB the value l6. 

Th coefficients obtained for the regression equations are shown in 
Table 12, which is laid out in a manner identical to that of Table 5* 

Insert Table 12 about here 

As in the case of 'feble 5 it is clear that NSTEPS is the most important 
of the three variables in predicting errors or success-latencies. Also 
the values obtained are comparable to those given in Table 5. In 
confines of the present paper it has not been possible to explore the 
possibility of a ;Joint analysis of addition and subtraction, with a par- 
ticular emphasis on process variables like NSTEPS, but this is a clearly 

indicated direction for future research. 

Again, as in the case of Table 5> the multiple correlation coeifi- 
cients shown in Table 12 indicate that the three independent variables 
are accounting for a good deal of the variation in the observed responoe 

proportions and success -latencies. 

Thbles 15-19 present predicted and observed proportions of errors, 
and other inf orma"!. ion identical to that given in Tables 4-11 for addition. 

Insert Teibles 15, l4, 15, l6, 17, l8 and 19 about here 

mt M M «»• •• • ^ 

The overall for subtraction exhibit a pattern very similar to those 
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Table 12 




Regression Coefficients for Subtraction 



Grade k Subtraction, Proportion of Errors 



Level 




Problems. 


Constant 


NSTEPS 


MAGDIF 


MAGSra 


R 


r2 


1 


5 


19 


-0.i^2 ’ 


0.06 


-0.03 


0.09 


0.73 


OM 


2 


11 


38 


-1.09 


0.19 


0.01 


, 0.02 


0.43 


0,x8 


3 




76 


-1.63 


0.12 


0.02 


0.09 


0.6l 


0.38 




1 Grade ! 


Subtraction 


, Success Latency 






Level 


Subjects 


Problems 


Constant 


NSTEPS 


MAGDIF 


MAGSUB 


R 




1 


5 


19 


6.8? 


0.^8 


-0.34 


-0.27 


0.62 


0.38 


2 


11 


38 


IM 


0.49 


O.Of 


0.11 


0.48 


0,23 


3 


^3 


76 


IM 


Oo 32 


0.06 


0.20 


0.64 


0.41 



Grade 5 Subtraction, Proportion of Errors 



Level 


Subjects 


Problems 


Constant 


NSTEPS 


MAGDIF 


MAGSOB 


R 


r2- 


1 


15 


38 


- 1.50 


0.15 


0.00 


0.08 


0.70 


0 .U 9 


2 


27 


57 


- - 1.98 


0.44 


0.00 


0.01 


0.80 


0.65 


3 


25 


76 


- 1.65 


0.40 


- 0.03 


0.01 


0.82 


0.68 


4 


9 


57 


-1.14 


0.20 


-0.01 


0.01 


0.68 


0.46 




Grade 5 ! 


Subtraction 


, Success Latency 






Level 


Subjects 


Problems 


Constant 


NSTEPS 


MAGDIF 


MAGSUB ' 


R 


r2 


X 


15 


38 


“1.77 


0.65 


0.12 


0.32 


0.73 


0.54 


2 


27 


57 


0.70 


0.99 


0.03 


0.01 


0.80 


0.64 


3 


25 


76 


- 1,91 


1.59 


-0.02 


0.04 


0.80 


0.64 


, 4 


9 


. 57 


2.58 


0.71 


-0.01 


0.00 


0.58 


0.34 







Predicted and observed proportions of errors and success-latency in 

fourth- grade subtraction, level 1 







Observed Predicted 


Observed 


Predicted 


2 


Rank 


Equations 


(1 - P^) (1 - P^) 


Latency 


Latency 


X 


1 


12-0 =•_ 


0.10 0.16 


2.70 


2c72 


0.13 


.2 


1 

It 

o 

1 


0.10 O.U 


1.70 


2.04 


0.08 


3 


_ - 3 = 11 


0.20 0.38 


7 .70 


4.58 


0.69 


k 


12 - 6 ss 


0.20 0.60 


3.00 


' 5.48 


3.33 


5 


lif - 3 = _ 


o.ifo 0.32 


4.70 


3.42 


0 .l 4 


6 


_ - 1 = 13 


0.1^0 0.29 


4.90 


5.03 


0.27 


T 


_ - 7 = 6 


0.4c . 0.70 


8.80 


6.37 


2.21 


8 


_ - 0 B 13 


0.40 0.19 


2.50 


3.55 


1.48 


9 


13 - _ = 7 


0.40 0.62 


7.00 


5.72 


1.00 


10 


11 ^ - _ 5 


0.40 0.77 


. 6,40 


5.59 


3^77 


11 


6 =5 


0.60 0.67 


7.80 


6.99 


0.12 


12 


12 - _ = 8 


0.60 0.50 


5.0. 


5.92 


0.18 


13 


11 -^= — 


0.60 0.49 


6.80 


5.68 


0.26 


Ik 




0.80 0.77 


4.00 


4.49 


0.03 


15 


15 ‘ 8 


0.80 0.65 


3.20 


. .5 J -1 


0.50 


16 


- 5 * ^ 


0.80 0.59 


2.70 


6.23 


0.94 


17 


- 10 rs 2 


0.90 0.79 


— 


M M 


0.38 


18 


1 ^ " «««, ~ ^ 


0.9c 0.72 


tm Mr> 


mm mm mm 


0.82 




11 - _ == 2 


0.90 0.80- 




— 


0.33 




16.65 (19 items) 


i X^( items < lO) 


= 16.65 (19 


items) 


= 3.84 



Table 



Predicted and observed proportions of errors and success-latency in 

fourth-izrade subtraction, level 2 







Observed 


Predicted 


Observed 


Predicted 


2 


Rank 


Equations 


(1 - Pi) 


(1 " Pi) 


Latency 


Latency 




1 


15 - 5 = _ 


0.05 


0.22 


2.70 


3.48 


1.98 


2 


17 - 7 = _ 


0.05 


0.2i^ 


2.60 


3.69 


2.25 


3 


12 - = 11 


0.05 


0.36 


2.60 


■ 4.09 


4.77 


k 


12 - 10 = _ 


0.05 


0.2ij- 


3.20 


3.58 


2.28 


5 


_ - 0 = 19 


0.09 


0.21 


4.00 


3.43 


0,90 


6 


_ - 0 = 13 


0.09 


0.19 


3.10. 


3.11 


0.71 


7 


16 - _ = l6 


0.18 


■ 0.37 


2.60 


4.25 


1.66 


8 


19 - _ = 10 


0.18 


C.3^ . 


4.30 


4.39 


1.28 


9 


11 - 10 = _ 


0.18 


0.24 


1.60 


.3-52. 


0.18 


10 


15 - _ = 6 


0.27 


0.54 


5.20 


5.16 


3.09 


11 


l6 - 10 = 


0.27 


0.25 


3.70 


■ 3.79 


0.03 


12 


20 - = 19 


0.27 


0.50 


3.00 


5.01 


2.21 


13 


17 - _ = l6 


0.27 


0.38 


2.60 


4-36 


0.54 


Ik 


15 » 10 = _ 


0.27 


0.25 


2.00 


3.74 


0.04 


15 


_ - 3 = 9 


6.27 


0.59 


6.90 


5.17 - 


4.46 


16 


20 - _ = 16 


0.27 


0.62 


4.60 


5.66 


5.80 


17 


20 - _ = 13 


0.36 


0.65 


5.90 


5.82 


3.80 


18 


19 - _ - 13 


0.36 


0.42 


6.30 


4.73 


0.16 


19 


19 - _ = 11 


0.36 


0.44 ' 


6.20 


4.83 


0.25 


20 


- 1 = 13 


0.36 


0.37 


4.40 


4.19 • 


0.00 


21 


11 - t = _ 


OM 


0.40 


4.00 


4.35 


0.l4 


22 


15 - _ = 8 


OM 


0.52 


4.20 


5.05 


0.20 


23 


17 - 8 = _ 


0.55 


0.43 


4.20 


4.72 


0.59 


2 k 


18 - = 13 


0.55 


0.4l 


4.80 


4.62 


0.80 


25 


13 - 2 = _ 


0.55 


0.20 


4.30 


3.21 


8.14 


26 


- 9 = 10 


0.55 


0.34 ' 


5.40 


4.39 


1.98 


27 


li<- - 3 = _ 


0.55 


0.21 


6.40 


3.32 


7.58 



Table 14 (continued) 



Rank 


Equations 


Observed 
(1 - Pi) 


Predicted 
(1 - Pi) 


Observed 

Latency 


Predicted 

Latency 


X^ 


28 


_ - 1 = 16 


0.64 


0.38 


4.10 


4.36 


3.06 


29 


— - 3 = 14 


0.64 


0.39 


5.60 


^.46 


2.69 


30 


_ - 5 = 5 


0.64 


0.60 


4.50 


5.22 


0.07 


31 


I 

II 
1 

o\ 

H 


0.64 


0.24 


4.90 


3.75 


9.64 


32 


--- - 3 = ,15 


0.64 


o.4o 


4.10 


4.52 


2,60 


33 


- 4 ^ 15 


0.64 


o.4i 


5.00 


4.62 


2.34 


34 


15 - = 9 


0.73 


0,51 


3.50 


5.00 


1.99 


35 


20 - 5 = __ 


0.73 


0.42 


4.80 


^.75 


4.24 


36 


11 - 3 = _ 


0.73 


0.37 


4.4o 


4.14 


5.95 


37 


5 = 11 


0.91 


0.40 


6.80 


4.51 


11.60 


38 


_ - 7 = 4 


0.91 


0.61 


7.80 


5.33 


4.10 



2 o 

A 104.03 (3Q items) X (items <.10) = 92.43 (37 items) = I.80 



yf' • 




Table 15 



Predicted and observed proportions of errors and success-latency in 



JLWtU. WLA- 



r\Y\ 



Igvsl 3 







Observed 


Pz'edicted 


Observed 


Predicted 


^2 


Rank 


Equations 


(1 ~ P^) 


•(1 - P^) 


Latency 


Latency 


X 


1 


19 - _ • 19 


0.01 


0.13 


2.00 


3.93 


6.65 




15 - _ = 15 


0.01 


0.11 


1.70. 


3.68 


5.53 


3 


13 - _ = 13 


0.01 


0.10 


1.70 


3-56 


5.04 


4 


8 - 4 = _ 


OoOl 


0.08 


1.90 


2.84 


4.4V 


5 


8 - S . 


0.01 


0.11 


1.30 


3.06 


7.62 


6 


11 - 9 = 


0.02 


0.33 


2,10 


4.64 


22.67 


7 


VO 

CVJ 

' II 

1 

VO 

cu 


0.02 


o.i6 


2.50 


4.36 


3.90 


8 


9 - 9 


0,03 


0.13 


1.30 


3.26 


7.95 


9 


8 - 2 = _ 


0.04 


0o06 


2.60 


2.57 


0.46 


10 


17 - 0 = 


0.04 


0.04 


2.30 


2.54 


0.05 


11 


_ - 1 = 21 


0.04 


o.i6 


4.50 


4.25 


2.91 


12 


9 - 7 = _ 


C.05 


0.12 


2.30 


3.24 


3.59 


13 


9 - ss 6 


0.05 


0.09 


2.90 


3.08 


1.26 


14 


_ . 0 = 2 


0.05 


0.04 


3.50 


2.24 


0.19 


15 


_ - 0 = 3 


0.05 


0.04 


2.70 


2.31 


0.13 


16 


10 - 6 = 


0.05 


0.22 


2.00 


4.18 


12.95 


17 


3 . _ =0 


. 0.06 


0.05 


2.50 


2.39 


0.13 


18 


_ - 1 = 0 


0.06 


0.05 


3.20 


2.32 


0.48 


19 


12 - 0 = _ 


O.'-Tf 


0.04 


1.80 


2.23 


0.40 


20 


12 - =6 


0. ;8 


0.28 


2.10 


4.62 


11.32 


21 


3 - =0 


. 0.08 


0.05^ 


2.50 


2.39 


0.76 


22 


4 - _ = 4 


0.08 


0.03 


2.7^ 


2.05 


4,10 


23 


‘ 25 - _ = 23 


0.08 


0.21 


3.60 


4.57 


2.65 


24 


10 - _ = 7 


0.09 


0cl8 


2.70 


4.09 


4.44 


25 


19 - 10 = _ 


0,09 


0.32 


2.30 


4.64 


12.07 


26 


5 - _ = 3 


OolO 


0.06 


2.80 


2.70 


1.84 


27 


11 - 4 


0.11 


0.17 


3.40 


3.97 


1.26 


28 


_ - ^ = 0 


0.11 


0.08 

s 


3.20 


2.90 


0.93 



o 







Tfible 15 (continued) 







Observed 


Predicted 


Observed 


Predicted 


P 


Hank 


Equations 


(1 - Pi) 


(1 - Pi) 


Latency 


Latency 




29 


19 - _ = 9 


0.15 


0.38 


3.20 


4.96 


11.46 


30 


- 4 « 10 


0,1T 


0.15 


5.20 


3.84 


0.14 


31 


11 « _ = 4 


0.19 . 


0.31 


4.50 


4.69 


3.63 


32 


_ - 2 = 13 


0.19 


0.15 


5.10 


3.95 


0.71 


33 


18 - 2 = _ 


0.20 


0.10 


4.40 


3.51 


1.53 


31^ 


13 - 3 - _ 


0.20 


0.10 


4.90 


3.33 


1.67 


35 


- 1 = 19 


0.21 


0.19 


5-20 


4.44 


0.12 


36 


12 - 4 « 

em»tm 


0.21 


0.18 


3.90 


4.03 


0.34 


37 


29 = 27 


0.23 


0.23 


3.20 


4.82 


0.00 


38 

j 


- 2 r: 19 


0.23 


0.27 


5.40 


4.95 


0.24 


39 


-1 = 8 


0.24 


0.08 


4.00 


3.13 


27.17 


li -0 


14 - _ = 5 


0.26 


0.42 


5.10 


5.14 


5.01 


' 41 


_ - 3 = 6 


0.27 


0.11 


4.10 


3.40 


19 o 64 


42 


- 2=4 


0.27 


0.08 


4.10 


3.08 


33.51 


43 


11 = 5 


0.27 


0.18 


3:80 


3.92 


.0.73 


44 


CVl 

cu 

ii 

1 

1 

VD 

CVl 


0.27 


0.27 


. 4.10 


4.90 


0.00 


45 


20 - 3 = _ 


0.28 


0.20 


4.00 . 


4.40 


2.23 


46 


_ - 10 = 2 


0.30 


0.37 


5 o 20 


4.84 


1.14 


47 


26 - _ = 21 


0.31 


0.31 


4.40 


5.04 


0.00 


48 


14 - « 6 


0.33 


0.38 


4.70 


5.01 


0.11 


49 


28 » 3 ____ 


0.35 


0.17 


5 .20 


4_.26 


5.82 


50 


1 

u 

0 

H 

1 

ON 

CU 


0.35 


0.41 


4.40 


5.26 


0.42 


51 


20 - = 12 


0.38 


0.43 


5.50 


5.38 


0.68 


52 


27 - _ = 19 


0.39 


0.57 


5.00 


6.13 


3.61 


53 . 


18 - _ = 8 


0.40 


0.50 


6.10 


5.53 , 


0.57 


54 


17 - _ = 15 


0.40 


0.24 


3. CO 


4.71 


2.01 


55 


20 - 7 * 


0.42 


0.33 


6.10 


4.93 


1.70 


56 


00 

c— t 

II 

1 

1 

CO 

CVl 


0.46 


0.53 


7.00 


5.83 


0 c 49 


57 


_ - 7 » 23 


0.46 


0.56 


6.20 


6.18 


0.96 


58 


18 - * 11 


0.47 


0.31 


6.20 


4.81 


1.65 


59 


17 - _ = 7 


0.47 


0.49 


4.30 


5.47 


0.02 





TaMe 15 (continued) 







Observed 


Predicted 


Observed 


Predicted 


2 


Rank 


Equations 


(1 “ V ^) 


(1 - Pi) 


Latency 


Latency 


X 


60 


1 

1 

VO 

11 

GO 


0.^3 


0.38 


6.00 


5.02 


1.48 


61 


U =.7 


0.53 


0.17 


6.90 


3.97. 


■ 13 .78 


62 


— - 6 = 19 


0.5^ . 


0.40 


6.20 


5.42 


2.07 


63 


- 9 s 16 


0.54 


0o53 


6.40 


5.83 


0.02 


Qir 


- 10 = 19 


0.58 


0.47 


5.90 


5.58 


1.10 


65 


II 

CM 

i 

1 1 


0.60 


0.17 


5.00 


4- .20 


19.78 


66 


_ - U = 8 


0,60 


0.18 


5.50 


4. .03 


18.35 


67 


- 10 = 9 


0.60 


0.38 


6.10 


4-. 96 


3.20 


68 


_ - 7 = 5 


0.60 


0.26 


8.20 


4.44 


8.74 


69 


- 10 = 6 


0.60 


0.35 . 


5.00 


4.77 


4.18 


70 


- 7 =21 


0.62 


0.34 


7.00 


5.11 


8.55 


71 


_ - 7 = 17 


0.62 


0.43 


6.00 


5.50 


3.58 


72 


_ - 9 - 19 


0.65 


0.56 


7.30 


6.01 


1.03 


73 


18-5 = _ 


0.73 


0,16 


5.40 


3.91 


36.64 


74 


- 6 = 6 


0.73 


0.23 


3^60 


4.30 


21.10 


75 


- 6=7 


0.80 


. 0.24 


4.20 


4.36 


25.88 


76 


22 - 8 = 


0.85 


0.39 


6.80 


5.19 


22.97 



2 

= !^1^.5.57 (76 items) X^(items < lO) = I36.3O (6l items) S = 1.68 




Table l6 

Predicted and observed proportions of errors and success-latency in 



jL wi^**gx'ctuc; ouuux uxwjLi^ u-ss v u. 







Observed 


Predicted 


Observed 


Predicted 


2 


Rank 


Equations 


(1 - Pi) 


(1 “ Pi) 


Latency 


Latency 




1 


It 

1 

1 

CO 

H 


0.03 


0.13 


1.70 


3.22 


1.91 


2 


19 - = 19 


0.05 


0.11 


2.20 


3.14 


0.70 


3 


17 - _ == 17 


0.05 


0.11 


1.50 


2.90 


0.70 


k 


II 

1 

1 

ON 


0.05 


0.13 


2 00 


3.34 


1.10 


5 


o 

CSJ 

II 

1 

) 

o 

(M 


0.05 


o.n 


1.90 


3.26 


0.70 


6 


25 - 1 = _ 


0.05 


0.10 


2.20 


3.1+1 


5 0.24 


7 


22 - =22 


0.05 


0.11 


2.10 


3.51 


0.35 


8 


- 0 * 22 


0.10 


0.11 


3.50 


3.51 


0.01 


9 


25 - 10 = . 


0.10 


0.38 


3.70 


5.18 


3.24 


10 


_ - i = 17 


0.15 


0.10 . 


4.00 


2,57 


0.70 


11 


18 - __ = 9 


0.20 


0.4l 


3.10 ■ 


.4.79 


3.64 


12 


- 5 = 18 


0.20 


0.39 


6.00 


5.^ 


1.45 


13 


_ - 4 * 17 


0.20 


0.34 


6.60 


• 5.48 


0.88 


Ih 


18 - 9 = _ 


0.25 


0.41 


3.20 


4.79 


2.11 


15 


19 - 2 « _ 


0.25 


0.08 


3.90 


2.23 


7.24 


16 


19 - _ - 9 


0.25 


0.46 


3.50 


5.11 


3.47 


17 


18 - _ = 13 


0.30 


0.24 


4.20 


4-.-01- 


0.36 


18 


_ - 1 = 15 


0.30, 


0.10 


4.60 


2.32 


9.74 


19 


- 2 = 19 


0.30 


0.26 


5.40 


5.09 


0.08 


20 


_ - 5 - 19 


0.30 


0.39 


4.70 


6.04 


0.31 


21 


1 

1 

o^ 

II 

\Jfl 


0,30 


0.43 


6.30 


5.87 


0.71 


22 


20 - =13 


0..35 


0.48 


4.30 


5.95 • 


1.36 


23 


1 

II 

1 

1 

ON 

H 


0.35 


0.13 


3.40 


3.08 


8.02 


24 


20 - 3 * _ 


0.35 


0.23 


4.40 


. 4.51 


1.53 


25 


_ - 3 = 22 


0,40 


0.13 


6.00 


3.81 


6.08 


26 


- 10 :« 32 


0.40 


0.46 


5.60 


5.1+7 


0.13 


27 


II 

on 

1 

1 ! 


0.45 


0.30 


6.10 


5.16 


2.19 



Table l 6 (continued) 



Rank 


Equations • 


Observed 
(1 - Pi) 


Predicted 
(1 - Pi) 


Observed 

Latency 


Predicted 

Latency 




28 


o \ 

II 

9 

1 


0.50 


0.40 


6.20 


4.81 


0.88 


29 


21 - 8 = _ 


0.50 • 


0.44 


7.40 


5.61 


0.12 


30 


21 - _ = 17 


0.50 


0.34 


■5.00 


5.48 


1.13 


31 


It 

CO 

1 

1 


0.60 


0.53 


6.60 


6.15 


0 . 4 l 


32 


^ - 5 = 13 


0.60 


0,24 


. 5.70 


4.01 


13.93 


33 


_ - 6 = 19 


0.60 


0.43 


7.10 


6.36 


1.15 


34 


22 - _ = 15 


0.60 


0.48 


5.60 


6.19 


0.57 


35 


1 

tl 

I 

OJ 


0.60 


0.40 


6.50 


. 5.78 


1.71 


36 


1 

il 

00 

1 

ir\ 

OJ 


0.60 


0.44 


6.80 


6.10 


0.97 


37 


1 

I 

VO 

II 


C.70 


0.58 


7.70 


6.71 


0.62 


38 


I 

1 

00 

II 


0.70 


0.53 


6.80 


6.63 


1.18 




X^(items <10) = 67. TO (37 items) = 1.64 



= 81.63 (38 items) 



Table I7 



Predicted and observed proportions of errors and success-latency in 



fifth-grade subtractior, level 2 







Observed 


Predicted 


Observed 


Predicted 


2 


Rank 


Equations 


(1 - P^) 


(1 - Pi) 


Latency 


Latency 




1 


kk - 22 = 


0.03 


0.22 


3.00 


4.55 


4.30 


2 


48 - 22 = 


0.13 


0.21 


4.70 


4.67 


0.70 


3 


47 - 20 = 


0.15 


0.2c 


k.90 


4.68 


0.34 


k 


58 - 35 = _ 


0.15 


0.26 


5.30 


4.72 


1.24 


5 


46 - 32 = 


0.18 


0.216 


5.00 


4.42 


1.66 


6 


48 - 36 = _ 


0.20 


0.28 


4 .40 


4.40 


1.47 


7 


48 - 37 = , 


0.20 


0.29 


3.60 


4.38 


1.66 


8 


1 

III 

CVJ 

on 

1 

ir\ 

ir\ 


0.20 


0.25 


4.40 


4.69 


0.25 


9 


58 - 33 = _ 


0.20 


0.25 


5 *80 


4.70 


0.29 


10 


48 - 20 = _ 


0.20 


. 0.20 


4.40 


4.71 


0.00 


11 


37 - 26 = _ 


0.24 


0.25 


4.10 


4.26 


0.00 


12 


37 - 25 = _ 


0.24 


0.24 . 


4.00 ' 


4.23 


0.00 


13 


^7 - 32 = _ 


0.24 


0.26 


5.30 


4.45 


0.07 


ih 


37 - 26 = _ 


0.24 


0.25 


3.30 


4.26 


0.00 


15 


44 - 20 = 


0.25 


0.21 


5.40 


4.59 


0.23 


16 


53 - 32 = _ 


0.25 


0.25 


4.60 


4.63 


0.00 


IT 


43 - 21 _ 


0.25 


0.21 


4.50 


4.54 


0.14 


18 


57 - 33 = _ 


0.25 


0.25 


5.30 


4.73- 


OoOO 


19 


56 - 30 = _ 


0.25 


0.24 


5.20 


4.75 


0.02 


20 


48 - 33 = 


0.27 


0.26 


5.90 


4.46 


0.00 


21 


41 - 30 = _ 


0.2"ir 


0.26 


4.30 


4.31 


0,01 


22 


38 - 27 = __ 


0.29 


0.25 


4.30 


4.28 


0.38 


23 


1 

1) 

m 

CVJ 

1 

VO 


0.31 


0.22 


4.10 


4.59 


0.86 


2 k 


48 - 31 = _ 


0.33 


0.25 


5.20 


4.50 


1.48 


25 


58 - 31 = _ 


0.38 


0.24 


4.40 


4 .80 


1.64 


26 


,56 - 32 = _ 


0.38 


0.25 


3.70 


4.72 


1.42 


27 


51 - 25 = _ 


0.38 


0.68 


4.90 


6.67 


6.80 


28 


36 - 21 = _ 


0.40 


0.22 


5.40 


4.33 


8.22 





Table 17 (continued) 



Rank 


Equations 


Observed 
(1 - P.^) 


Predicted 
(1 - Pi) 


Observed 

Latency 


Predicted 

Latency 




29 


50 - 24 =• _ 


0.40 


0.68 


6.30 


6.66 


6.90 


30 


50 - 24 = 


0.44 


0.68 


4.80 


6.66 


4.10 


31 


55 - 34 = _ 


0.44 


. 0.26 


4.00 


4.65 


2.68 


32 


40 - 18 = __ 


0.45 


0.66 


6.50 


6.48 


3.80 


33 


40 - 18 = _ 


0.50 


0.66 


6.40 


6.48 


2.18 




44 - 21 = ___ 


0.56 


0.21 


3.10 


4.57 


11.92 


35 


57 “ 28 = __ 


0.56 


0.69 


5.40 


6.80 


1.13 


36 


53 - 24 = _ 


0.56 


0.67 


7.30 


6.75 


0.82 


37 


40 - 19 = _ 


0.56 


0.66 


6.10 


6.46 


0.72 


38 


4C - 17 = _ 


0.56 


0.65 


6.60 


6.50 


0.54 


39 


31 - 16 = _ 


0.60 


0.66 


6.80 


6.25 


0.76 


ko 


41 - 15 = 


0.60 


0.64 


7..3O 


6.57 


0.11 


in 


41 - 15 = _ 


0.63 


0.64 


7.10 


6.57 


0.01 


k 2 


42 - 13 = _ 


0.63 


0.62 


6.90 


6.64 


0.00 


43 


43 - 14 = _ 


0.63 


0.63 


6.^ 


6.65 


0.00 


kk 


54 - 26 = _ 


0.63 


0.68 


7.30 


6.75 


0.22 


45 


52 - 24 = _ 


0.65 


0.67 


8.30 • 


6.72 


0.04 


46 


31 - 18 = _ 


0.69 


0.67 


6.20 


6.21 


0.05 


47 


50 - 24 = 


0.70 


0.68 


6.80 


6.66 


0.06 


48 


32 - 15 = _ 


0.71 


0.65 


. 5.50 


6.30 


0.66 


49 


^3 - _ 


0.75 


0.63 


7*70 


. - 6^5 


1.32 


50 


42 - 15 = _ 


0.75 


0.63 


6.70 


6.60 


1.16 


51 


41-27 = 


0.76 


0.71 


7.00 


6.34 


0.49 


52 


43 - 24 := 


0.78 


0.69 


6.00 


6.46 


1.71 


53 


44 - 27 = _ 


0.78 


0.70 


5.70 


6.43 


1.20 


5 h 


43 - 26 = _ 


0.82 


0.70 


6.70 


6.42 


3.21 


55' 


53 - 28 = _ 


0.85 


0.69 


^ mm wm 


mm mmm M 


2.31 


56 


55 - 29 = _ 


0.85 


0.70 


7. 50 


6.72 


2.25 


57 


51 - 23 = _ 


0.95 


0.67 


w «• 


— 


7.21 



! . ' 

! 

S ' 

t ' 



o 2 

X® = 90.67 (57 , items) X^(items < 10) -= 78.?6 (56 items) S = 0.6U 



o 

ERIC 





Table l8 




Predicted and observed proportions of errors and success- latency in 

fifth-grade subtraction, level 3 




Rank 


Equations 


Observed 
(1 - Pi) 


Predicted 

(1 “ Pi) 


Observed 

Latency 


^edicted 

Latency 


1 


49 - 20 = _ 


0.03 


0.10 


3.40 


3.12 


2 


47 - 24 = 


0.05 


0.16 


2.70 


3.41 


3 


46 - 21 = 


0.07 


0.13 


3.90 


3.25 


k 


57 - 32 = _ 


0.07 


0.18 


5.20 


3-71 


5 


49 - 25 = _ 


0.07 


0.15 


4.10 


3.‘^'^ 


6 


I 

II 

CVI 

I 

VO 


0.07 


0.17 


3.70 


3-‘^3 


7 


57 - 35 = _ 


0.10 


0.22 


3.00 


3.90 


8 


47 - 21 = _ 


0.10 


0.12 


3.00 


3.23 


9 


45 - 20 = _ 


0.10 


0.13 


2.70 


3.20 


10 


49 - 27 = _ 


0.10 


0.18 


2.20- 


3.56 


11 


67 - 30 = _ 


0.10 


0.09 


2.90 


■ 3-39 


12 


51 - 20 = _ 


0.10 


. 0.09 


2.90 


3-08 


13 


58 - 27 = _ 


0.10 


0.09 


3.20 


3.30 


Ik 


, 59 - 21 = „ 


0.10 


0.06 


3.20 


3.00 


15 


■ 56 - 25 = _ 


0.10 


0.11 


3.10 


3.30 


16 


59 - 21 = _ 


0.10 


0.06 . 


3.90 


3.00 


17 


47 - 23 = _ 


0.13 


0.15 


4.40 


3.35' 


18 


59 - 37 = _ 


0.20 


0.23 


3.50 


3.99 


19 


5^ * 31 - 


0.20 


0.19 


2.80 


3.71 


20 


47 - 22 = 


0.20 


0.l4 


3.80 


3.29 


21 


53 - 30 - _ 


0.20 


0.18 


1.90 


3-67 . 


22 


66 - 33 = _ 


0.20 


0.12 


2.50 


3.60 


23 


62 “ 27 " __ 


0.20 


0.43 


6.70 


6.53 


24 


60 - 23 = _ 


0.20 


0.33 


8.10 


6.26 


25 


69 - 38 = _ 


0.20 


0.15 


2.30 


3.85 


26 


50 - 12 = _ 


0.20 


0.25 


7.00 


5.78 


27 


■ 50 - 12 = _ 


0.20 


0.25. 


4.80 


5.78 



X 

0.80 

0.88 

0.55 

1.23 

0.87 

1,07 

0.83 

0.06 

0.07 

0.43 

0.01 

0.00 

0.01 

0.10 

0.00 

0.10 

0.02 

0.05 

0.01 

0.36 

0.02 

0.31 

1.08 

0.38 

0.09 

0.06 

0.06 



Table l8 (continued) 












Observed 


Predicted 


Observed 


Predicted 


0 

X*" 


Rank 


Equations 


(1 “ Pi) 


(1 - Pi) 


Latency 


Latency 


28 


62 


- 


26 = _ 


0.20 


0.37 


7.00 


6.41 


0.59 


29 


56 


- 


17 = 


0.20 . 


0.27 


5.00 


5.97 


0.11 


30 


44 


- 


33 = _ 


0.24 


0.34 


3.20 


4.03 


3.15 


31 


50 


- 


21 = _ 


0.27 


0.43 


5.50 


6.34 


1.61 


32 


34 


- 


22 = _ 


0.27 


0.26 


4.20 


3.55 


0.06 


33 


30 


- 


15 = _ 


0.28 


0.59 


’ 3.00 


6.36 


28.22 


3^^ 


4? 


- 


30 = _ 


0.30 


0.25 


4.60 


3.79 


0.99 


35 


^5 


- 


21 = 


0.30 


0.14 


2.50.’ 


3.27 


2.18 


3e 


50 


- 


24 = _ 


0.30 


0.50 


4.50 


6.52 


■ 1.55 


37 




- 


36 = _ 


0.30 


0.24 


3*80 


3.96 


0.23 


38 


51 


- 


26 = _ 


0.30 


0.53 


7.80 


6.63 


2.08 


39 


44 


- 


16 = 


0.30 


0.4l 


5.50 


6.14 


0.46 


ko 


39 


- 


26 = _ 


0.31 


0.27 


5.10 


3.70 


0.66 




49 


- 


30 = _ 


0.35 


0.22 


3. '60 


3.75 


.6.75 


42 


49 


- 


35 = _ 


0.37 


0.31 


. 4.80 


4.06 


0.94 


43 


• 49 


- 


34 = _ 


0.37 


0.29 


4.60 • 


4.00 


1.81 


44 


49 


- 


30 = _ 


0.39 


0.22 


4.30 


3.75 


11.89 


45 


50 




29 = _ 


0.40 


0.61 


6.50 


6.84 


2.77 


46 


47 


- 


18 = __ 


0.40 


0.4l 


7.30 


6.21 


0.00 


47 


59 


- 


30 = _ 


0.40 


0.14 


4.30 


.. 3*55 


5.92 


48 


42 




16 = _ 


0.40 


0.44 


5.40 


6.18 


0.05 


49 


56 


- 


19 = _ 


0.40 


0:30 


7.20 


6.09 


0.22 


50 


62 


- 


29 = _ 


0.40 


0.43 


6,90 


6.60 


0.02 


51 


55 


- 


16 = 


0.40 


0.26 


4.00 


5.93 


0.51 


52 


50 


- 


12 = 


0.40 


0.25 


3.60 


5.78 


0.62 


53 


65 


m> 


27 = _ 


0.40 


0.34 


. 7.60 


6.4i 


0.07 




49 


•r 


30 = _ 


0.47 


0.22 


4.60 


3.75 


23.85 


55 


46 


- 


17 = _ 


0.47 


0.40 


6.00 


6.17 


0.29 


56 


41 




16 = 


0.47 


0.45 


7.40 - 


6.20 


0.02 


5? 


52 




26 = _ 


0.50 


0.51 


5.80 


6.61 


0.01 












Table IB (continued) 



Bank 


Equations 


Observed 
(1 - Pi) 


Predicted 
(1 " Pi) 


Observed 

Latency 


Predicted 

Latency 


X^ 


58 


45 - 19 = __ 


0.50 


0,46 


6.50 


6.31 


0.07 


59 


li-0 - 21 = _ 


0.52 


0.58 


5.20 


6.54 


0.97 


60 


M 

1 

U) 

II 

! 


Oo53 


0.38 


3. 50 


6.02 


1,42 


61 


^3 - 19 - _ 


0.53 


0.49 


6.50 


6-35 


0.12 


62 


37 - 19 = _ 


0.58 


0.58 


6.60 


6.47 


0.00 


63 


k 6 - 27 


0.58 


0.62 


5.50 


6.79 


0,65 


6 k 


3^ - 17 = _ 


0.59 


0.56 


6.50 


6.39 


0.25 


65 


53 - 29 = _ 


0,60 


0.57 


9.50 


. 6.78 


0.07 


66 


54 - 26 = _ 


0.60 


0.48 • 


6.50 


6.57 


0.83 


67 


^3 - 17 = 


0.60 


0.44 


7.50 


6.23 


1.50 


68 


- 18 = _ 


0.60 


0.45 


■6.30 


6.27 


1.35 


69 


51 - 25 = _ 


0.60 


0.50 


8.10 


6.57 


0.56 


70 


46 - 28 = _ 


0.65 


0.64 


7.-20 


6.85 


0.00 


71 


32 - 13 - _ 


0.66 


0.52 


6.00 • 


6.20 


‘ 5-98 


72 


37 - 19 = _ 


0.69 


0.58 


7.20 


,6.47 


3-64 


73 


I 

II 

{>• 

OJ 

1 


0o72 


0.64 


6.90 


6.81 


2.00 


Ik 


51 - 28 = _ 


0.73 


0.57 


6.90 


6.75 


1.57 


75 


55 - 29 = _ 


0.73 


0,54 


4.80 


6.74 


2.34 


76 


31 *• 17 = _ - 


0.78 


0.62 


6.50 


6.46 


7.15 



= 137*3^ (76 items) ' (items < 10) = 73-39 (73 items) = 1.25 





Table 19 



Predicted and observed proportions of errors and success latency in 

jaj ^ «•- _*i I, 

ixxun-graue suuuraci>xon, xevex m* 







Observed 


Predicted 


Observed 


Predicted 


0 


Rank 


Equations 


(1 - P^) 


(1 - Pi) 


Latency 


Latency 


y ? 


1 


29 - 5 = HK 


0.04 


0.18 


3.60 


4.56 


1.52 


2 


29 - 24 = M 


o.o4 


0.29 


1.20 


4.67 


3.61 


3 


49 - K = 43 


0.06 


0.21 


3»20 


5.17 


1.20 


4 


36 - 5 = MK . 


0.07 


0.16 


4.40 


4.53 


0.43 


5 


43 - 21 = R2 


0.07 


0.16 


3.10 


3.87 


0.40 


6 


42 - 5 = R7 


0.08 


0.22 


5.30 


5.20 . 


1.29 


7 


25 = 29 


0.11 


0.25 


2.60 


5.26 


0.92 


8 


38 - 12 = M6 


0.11 


0.13 


3.50 


3.85 


0.03 


9 


i 

1 

00 

CVJ 

tl 

cu 


0.14 


0.25 


2.50 


5.27 


0.44 


10 


37 - M = 31 


0.14 


0.17 


3.50 


4.53 


0.03 


11 


41 = 49 - 


0.17 


0.22 


. 4.40 


5.18 


oa8 


12 


37 = 42 - 


0.22 


0.41 


5.4o 


6.61 


1.33 


13 


_ - 5 = 23 


0.22 


0.26 


7.70 


5.27 


0.06 


14 


37 - 5 = HK 


0.22 


0.11 


4.20 . 


3.81 


1.22 


15 


32 - _ = 25 


0.22 


0.4; 


6.10 


6.67 


2.16 


16 . 


37 - K = 32 


0.25 


0.23 


3.70 


5-23 


0.02 


17 


40 - sr 26 - 25 


0.25 


0.71 


5.90 


8.71 


12.60 


18 


43 - 5 = _ 


0.29 


0.30 


7.70 


■' 5.90 


0.01 


19 


- 6 = 24 


0.33 


0.47 


5.40 


6.68 


0.64 


20 


33 = _ - 6 


0.33 


0.23 


5.60 


5.22 


0.51 


21 


54 - 17 =. P7 


0.33 


0.26 


6.30 


5.20 


0.27 


22 


43 - 8 = _ 


0.33 


0.33 


6.90 


5.91 


0.00 


23 


_ - 5 = 29 


0.42 


0.24 


7.00 


5.24 


2.0)+ 


24 


_ - 9 - 25 


0.42 


0.48 


5.60 


6.67 


0.16 


25 


32 - = 27 


0.42 


0.45 


5.20 


6.66 


0.05 


26 


27 . 


0.42 


0.45 


6.20 


6.66 


0.04 


27 


_ - 7 = 19 


0.43 


0.49 


8.20 


6.71 


0.10 


28 


27 = - 4 


0.43 


. 0.45 


7.80 


6.66 


0.01 



Table 19 (continued) 



Rank 


Equations 


Observed 
(1 - P^) 


Predicted 
(1 - 


Observed 

Latency 


Predicted 
Latency • 




29 


28 - KR = 7 


0.43 


0.37 


4.30 


5.36 


0.09 


30 


26 =: - 7 


0.44 


0.46 


8.00 


6.67 


0.01 


31 


39 - bn = 7 


0.44- 


0.42 


5.80 


5.36 


0.02 


32 


_ = 35 - 8 


0.44 


0.46 


8.30 


6.66 


0.01 


33 


42 - 4o = 38 - 


0.44 


0.69 


4.10 


6.80 


2.65 


3h 


35 = 4l - 


0.50 


,0.42 


4.50 


6.62 


0.29 


35 


_ - 25 = 37 - 36 


0.50 


0.44 


5.90 


6.67 


0.18 


36 


38 - 9 = _ 


0.50 


'-'O? 


4.10 


5.95 


1.19 


37 


28 = _ - 5 


0.50 


0.24 


6,20 


5.25 


4.29 


38 


=: 26 - 9 


0.56 


0.51 


8.90 


6.73 


0.06 


39 


26 - = 30-6 


0.56 


0.67 


6.90 


8.09 


0.53 


40 


- 26 = 32 - 30 


0.57 


0.64 


• 8.60 


6.80 


o.i4 


4l 


27 = _ - 8 


0.57 


0.46 


8.80 


6.66 


0.33 


42 


_ =: 34 - 8 


0.57 


- 0.47 


9.20 


6.67 


0.31 


^3 


1 

1 

VO 

CVI 

II 

H 

1 


0.57 


0.57 


4 . 60" 


6.79 ■ 


0.00 


44 


27 - 24 = 33 - _ 


0.57 


0.63 


6.50 


6.79 


0.09 


45 


_ = 32 - 5 


0.57 


0.45 


7.50 


6.66 


0.42 


46 


- 24 = 6 


0.57 


0.61 


8.00 


6.78 


0.06 


47 . 


34 - 31 = - 26 


0.58 


0.54 


8.50 


7.35 


0.09 


48 


_ - ^^3 - 7 


0.67 


0.42 


&.50 


6.61 


2.9c 


49 


45 - FG ^ 3 


0.67 


0.48 


6.50 


'5.38 


1.66 


50 


42 - 15 = K7 


0.71 


. 0.28 


6.00 


5.25 


6.48 


51 


_ = 32 - 6 


0.75 


0.46 


9.00 


6.67 


4.12 


52 


_ - 27 = 36 - 32 


0.78 


. 0.87 


- -- 


— 


0.73 


53 


53 - 17 m 6 


0.83 


0.36 


7.10 


5.91 


11.78 


54 


23 - 8 = MH 


0.86 


0.40 


9.60 


‘6.02 


6.19 


55 


er - _ = 32 - 6 


0.86 


0.88 


8.40 . 


10.19 


. 0.05 


56 


25 - 21 = - 33 


0.89 


0.80 . 


8 .80 


8.20 


0.48 


57 


2l|. - =16 


0.93 


0.51 


--- 


— 


5.01 



« 81A3 (55 items) X^( items < 10) = 57.05 (55 items) 2.93 



2 

obtained for addition. In the case of Tables 17 and 19 the X values, 
including the largest individual itein contributions are ^ust signii leant 
-l-Viia m 1 onrial ftinH «*! jm*i iM ofint. at. all in tha CES6 of Tfibl© 13* 

Tables 13-19 also show the predicted and observed success-latencies 
for the subtraction data. Again the statistic is given at the bottom 

of each table. Applying the same interpretation as before to this statistic, 
we may look at the value of S for each table as an estimate of the stan- 
dard deviation of the approximately normal distribution of errors. For 
these seven tables we find the values of S to be I.96, 1.3^> 1*30» 1.28, 
0.80, 1.12 and I.7I respectively, and it is, reasonable to say that errors 
of prediction greater than about I.50 seconds should not occur very often. 
The observed success-latency values have a range from slightly more than 
5 seconds (Table 1?) to more than 8 seconds (Table 19), and consecuently, 
a model with errors that have an approximately normal distribution with a 
standard deviation of about 1.5 seconds yields meaningful and useful pre- 
dictions. These results are very comparable to those found for addition, 
llie same is true of the measure of average percentage error, which is 
25.756, 23.056, 24.^, 26'.3^, 10.95^, 17.1^, and 2 ^. 0 $ for Tables 13-19 
respectively. 

Without making an exact statistical comparison it still seems clear 
that the approximate measure, of fit we have reported for the success- 
latencies in subtraction reflect a better fit to the data than do the 
measui’es for predicted response proportions. The predictions of 
response propoi t. ons still leave a lot to be desired. The predictions 
of success-latencies seem to reflect more regularly the observed rankings 
of latencies, aven though this apparent difference in favor of latency 
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predictions is not well reflected in the multiple correlation coefficients 
of Table 12. 

Inspection of the tables for subtraction confirms the intuition that 

subtraction problems of the form - n = p are not relatively as diffi- 

c\ilt as the same form is in the .case of addition. No doubt the reason 
for this is that a single simple transformation converts such subtraction 
problems into the easiest sort of addition problem, p + n = ^ 

It should be noted that Table 19 includes problems using letter 
variables as well as blanks, and it is interesting to note that prob?.ems 
using letter variables are the six easiest problems in the table in terms 
of response errors, although the same six problems do not have the shortest 
latencies.^ The format of these problems with letter variables was of the 
following sort: 



^5 


29 




37 




29 


-FG 


- 5 




- K 




-2k 


3 FG = 


HK 


HK 


32 


K = _ 


M M = 



The ease of handling algebraic notation is also confirmed by some other 
unpublished experiments conducted in the Institute several years ago 
with first- and second-grade children. 

Mult ’■Plication-grade four . The two sets of problems considered 
each contained twenty exercises, as in the case of addition and subtraction. 
The two sets concentrated on a review of multiples of 4 and 5> with 
the second factor ranging from 0 to 12. The problems occurred in 

the three forms, m x n = , m X = p’, and X n = p. Unlike the 

addition and subtraction analyses covered in the previous pages NSTEPS 



36 




was not considered as a variable because in all problems only one operation 
was involved. To see if transformations as described in the theory section 
defined a significant variable, we treated each of the three equational 
forms as an independent variable vdiich took on the value 1 if the problem 
was in the given form and 0 if it were not . The other two independent 
variables used were the larger factor (LARGER) and smaller factor (SMAUER) 
that yielded the product. In the case of squares (li- X 4 and 5 5) 

values of the two factors were equal. Table 20 presents the regression 
coefficients for the five variables considered with proportion of errors 
and success-latency as dependent variables. 



Insert Table 20 about hers 



Again we found that the linear. regression model does well at prediotxng 
errors and success-latency from a small nomber of variables. The only 
equation-form varicdjle that significantly affected the regression line 

was the canonical form a x b = and the negative coefficients of this 

variable indicate that problems of this form are easier tlian'probiems cf 
the form x b = c or a x = c, a finding well in keeping with 

intuition. 

2 

When all forty problems were considered the overall X = 113 »29, 
but -tfhe deletion of three problems each with a component great'^r 

than 10 dropped the total for the remaining 37 problems to X =» 41.43. 

The three problems dropped from the analysis were 4 X 10 - , for wh^ch 

the predicted error proportion was much higher than observed, and 4 X — - 
X 4 = 48 for which the predicted error proportions were much 



and 







K 

o 

tj 




O 00 

• • 

o o 



(C5 

X 



I 

I 

I 

I 



( 

I 

I 

I 







C) 










II 








fl 




1 


1 




0 


1 


1 


1 




•H 


1 


1 


1 




+5 




1 


i 




d 


X 








0 










•H 


d 








rH 










ft 










•H 










43 


I 








rH 


1 








I 


11 


cr\ 


OD 








CVl 


00 




(U 


cQ 


• 


■ 




'd 




0 


0 




d 


X 


8 


1 




?4 










t^D 


d 








1 




















43 


u 










Q) 






0 


d 


rH 


rH 


CO 


CM 


0 


H 


0 


00 




<+-4 


cd 


• 




Q> 




B 


0 


.0 


H 


u 


CQ 






rO 


0 








d 










EH 




Q> 


0 


00 




CO 


W 


rH 


rH 




4 > 


U 


■ , 


• 




C. 


d 


0 


0 




Q> 










•H 










0 


43 








•H 


C 5 


V£) 


CVl 






d 




0 






43 


■ 


■ 




Q> 


CO 


pH 


H 




0 


d 


t 






0 


0 










0 








a 










. 0 


CO 








•H 


43 








(0 


0 








CO 


0 ) 


C^ 


C^ 




0 ) 


•n> 


- 4 * 






. 


rO 








w 


d 








Q> 


CO 








« 









CO 

B 

(D 



H O O 



43 

d 



43 




H 


Cl 




1 


(L> 




CO 


*d 


la 


CO 


fl 




Q) 


0 ) 


0 


0 


ft 


iH 


0 


0 ) . 




d 


0 




CO 





lower than observed. It is not difficult to analyze why these three 
problems probably deviated greatly from the predicted values. In general 
4 X 10 = allows use of the simple algorithm a X 10 = aO. We would 
exoect the same low error finding for 5 X 10 = » but this problem did 

not occur in the two sets. 'The ‘problem 4 X = 4 turned out to be the 

first multiplication problem presented, and as mentioned in the discussion 
of addition, there is evidence of a warm-up effect which affects response 

to the first problem of the day. The problem x 4 = 48 is the only 

problem of the set for which uh^ initial factor is both 12 and also 
the answer to be found. The for comparing observed and pre(^cted 

latency was quite low. The obtained value, ~ .62, indicates that most 
prediction errors were definitely ,le so than 1.0 second. 

Multiplication t ables— grades three , four, five and six. Toward the 
end of the school year we decided to run the 100 one- digit multiplication 
problems of the form a X b = to see how well a structural model would 

predict response behavior. Previous investigations of performance on these 
basic multiplication facts are not as numerous as we had expected, and the - 
kind of regression model applied here has not been previously used, as 
far as we know. The first point to note is that for all four grades, the 
response performance was extremely good. The error rate was 8.0 per cent 
for third-gr,ir:e children and 3.2 per cent for sixth-grade children, with 
the fourth and fifth grades falling between these two bounds.; Consequently 
our anr.Vvsis in this case is restricted entirely to success^latencies , 
because the form of the equations was constant in the 100 problems, we 
have restricted our regression to the two factors, SMALLEIR and LARGEIR, 
already used in analyzing fourth- grade multiplication. The regression 
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coefficients, multiple correlation and statistic S for each grade are 
shown in Table 21. There are several observations to be made about this 

Insert Table 21 about here 

table. In the first place, for all four grades the multiple correlation 

♦ 

R is extremely high, indicating that the two variables are giving a good 

account of the data. This inference is supported by the small values of 
2 

S , which are the lowest values reported for any of the sets of data 
analyzed in this paper. It is also apparent from the values of the 
regression coefficients that the magnitude of the smaller factor is more 
important than that of the larger factor. Thus, for example, on the 
average it takes longer to say what 1X9 i? than to say what if X 5 
is. Finally, with analysis for four grades before us, it is natural to 
ask whether we can find evidence of development from one grade to another. 
Development is most evident in the monotonically decreasing values of the 
constant, which reflect an increase in speed of response with age- Tn 
the regression model for latencies, the constant enters in a direct additive 
way. The decrease from I. 7 I seconds in the third grade to 1.33 seconds 
in the sixth grade is not surprising. What is surprising is_ that the 
coefficients of the two factors do not show a corresponding monotonicity 
with age. This lack of monotonicity complicates considerably the task 
of constructing a model of developmental processes and their effects on 
arithmetic performance. 

Figures 5 and 6 show the predicted and observed success- latency 
curves for the third and sixth grades respectively. The 100 problems 

Insert Figures 5 and 6 about here 
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Figure 5 . Predicted and ob.served success-latencies of third-grade students for the multiplication 
. tables. 






Table 21 

Linear-regression coefficients 
for the multiplication tables 



Grade 


Subjects 


Constant 


Larger 

Factor 


Smaller 

Factor 


R 


e2 


s2 


3 


2 k 


1.71 


0.06 


0.30 


0.86 


0,74 


0,22 


k 


56 


1.52 


0.07 


0.28 • 


0.85 


0.73 


0.22 


5 


20 


1.38 


0.09 


0.29 


0.78 


0.63. 


O.U2 
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32 


1.33 


0.06 


0.19 


0.82 


0.68 


o.u 



OS 




(SONOOBS) AONBXVI SSBOOnS 



S 

•H 

-P 

cd 

o 



St 

•p 



B 

Q> 



txt 

-p 

c ; 

0) 

S 

€0 

Q) 

»d 

cd 

!o 

P 

X 

•H 

CQ 

<P 

o 

CO 

Q ) 

•H 

0 
C 
Q) 
P 
cd 

fH 

1 

(0 

CO 

Q ) 

o 

o 

d 

CO 

Q ) 

> 

u 

0) 

QQ 

•§ 



Q) 

P 

O 

•H 



CO 

O 



H 
Q) ;0 
^ <d 
p 



£ 




Table 22 



Linear-regression coefficients for transformation, operation and 
memory steps in fourth-grade addition 





Constant 


Transformation 


Operation 


Memory 


K 


r2 


Errors 


-1.29 


0.20 


0.00 


0.39 


0.73 


0.53 


Latencies 


2 ,^ 


0.58 


0.00 


0.75 


0.69 


0 A& 



are rank ordered according to success-latency on the abscissa, and thus 
the observed data define a relatively smooth monotonically increasing 
function. The predicted curve is determined for each grade level by 
the three estimated coefficients given in Table 21 and the two given 
factors of each multiplication problem. Considering the wi.de range of 
latencies found in each figure, running from 1.5 to seconds in the 
third grade, and from 1.4 to 4.0 seconds in the sixth grade, we feel that 
the predicted curves are fitting the observed data quite well. For those 
readers accustomed to looking at smooth predicted learning curves that 
are essentially exponential in form, we emphasize that the predictive 
task is different and rather more difficult, as we move not from like trial 
to like trial, but from a problem-item \T±th a particular structure to 
another problem- item with a distinct . structure . 

Analysis of jttie factors in NSTEPS. As we promised in the theoretical 
discussion of the second section, we now present a preliminary analysis 
of breaking up the single variable NSTEPS into its three components of 
transformation , operation and memory . There is one slight difference in 
the analysis presented here from the definition given in the second sec- 
tion. Transformation steps always were jither 0 or 1, never 2. With 
this exception, the analysis was entirely ba':ed on the ea'f’ier definitions. 
The data used in the first analysis are those already rey ,rted in Tables 
5 and 6, but -aout the first item of each set of problems deleted. 

Thus this first analysis is in terms of 80 fourth- grade addition problems. 
The results are shown in Table 22. In the case of both errors and 



Insert Table 22 about here 
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success-latencies it is in^ortant to observe that memory is the most 

important variable, while operation plays no role. Moreover, in both 

cases we get nearly as good a fit sin?)ly by using memory as the single 

variable. In the case of errors the difference in the multiple correlation 

R occurs only in the third decimal, .726 rather than .731, and in the 

case of latency .677 rather than .688. The arxd S values that 

come from using the coefficients of Table 22 are high but are not out of 

2 

line with those reported earlier. In particular X = ^'7.7, and if we 

2 

delete the 12 extreme items having individual X 's greater than 10, 

2 ^,^2 
X = 170«3> for the remaining 68 items. The statistic S = l.lj-2,. which 

yields an estimate of I.I9 for the standard devi.ation of the errors in 

prediction. What is particularly worth noting in a comparison of Tables 

5 and 22 is that the correlation for fourth-grade addition (block 1, level 

3) is lower than the correlation for the combined data of Table 22. • (For 

the problems of this block, see Table 5«) 

A second, somewhat different analysis was performed on a set of 19 

problems that, together with the initial problem omitted, in the analysis, 

formed one day’s exercises on fourth-grade addition, block 3 9 level 1|-. 

These I9 problems are among the j6 already analyzed in Table 10. Tlie 

departures from the earlier definitions of the components of NSTEPS 

were these. First, because the problems were all of the form 

ab + cd = + ef or ab + cd = ef + , the number of transformations 

was the same for all problems and therefore was omitted as a variable. 

Second, the operations of addition and' subtraction of single digits were 

treated as separate variables. Third, the number of digits in memory was. 

ejq)anded to include all digits used in obtaining a solution, including 



kl 





those presented in the problem, those that occurred as partial solutions, 
and those that were present in the response, ihe three variables consid- 
ered were, therefore, number of addition operations (OPl;, number of sub- 
tractions operations (0P2) and nunber of digits processed (MEMORY). 

Table 23 presents the regression coefficients for the three variables 
found, with proportion of errors and success-latency as dependent variables. 

Insert Table 23 about here 

The very high correlations for both errors . and latencies warrant a closer 
look at the results. 

For the data entering this analysis the mean number of addition 
operations was 2,4, the mean number of subtraction operations was 1.8 and 
the mean number of digits processed was 8 . 7 . It wou.ld appear that the 
number of addition operations has a much smaller effect on errors than the 
number of subtraction operations. Neither of these two variables has a 
significant effect on success-latency. Figure 7 presents the observed 
and predicted proportion of errors as a function of ranked difficulty. 

With the exception of problems 6 and 8, the observed end predicted curves 

M w M « * •• * •• • • • » « ^ •• “ 

Insert Figures 7 and 8 about here 

are .quite similar. Figure 8 presents observed and predicted success- 

latencies as a function of observed latency rank. Once more we find the 

general shapes of the observed and predicted curves quite similar. Figure 

9 is s scatter plot of observed versus predicted errors. Figure IG is a 

similar scatter plot of observed versus predicted success-latency. If 
* 
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Figure 7* Predicted and observed proportions of errors on 
fourth-grade addition analyzed in terms of tiiree 
process variables. 



Table 23 



Linear- regress ion coefficients for OPl, 0P2 and MEMORY steps in 

fourth-grade addition 





Constant 


OPl 


0P2 


MEMORY 


R 




Errors 


-2.65 


0.06 


0.25 


0.25 


0.89 


0.79 


Latencies 


-0.42 


0.00 


0.00 


0.77 


• 0.86 


0.73 
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ACCORDING TO SUCCESS LATENCY 



Figure 6. 



Predicted and observed success-latencies for 
fourth-grade addition analyzed in terms of three 
process variables. 




Insert Figures 9 about here 

all the points in the two plots fell on the 45° straight line the pre- 
dictions wo-old he perfect. The deviations of the points from this line 
are a measure of the goodness of fit of the model . 

5 , Discussion , 

In this final section, we shall not atteii®t to summarise in system- 
atic form the results reported in the previous section. It is our own 
feeling that the results establish clearly enough the real possibility 
of analyzing and predicting in terms of meaningful variables the response 
and latency performance of children who are .solving arithmetical problems. 

As we have already stated, the predictive results reported here have been 
good enough to be practically useful, but they are incbir®lete enough to 
present a challenge to anyone interested in systematic psychological theory. 

From a psychological standpoint, the most suggestive single finding 
is probably the importance of the process variable NSTEPS, or of its 
congionent variables, particularly memory, in all the relev^t. analyses . 

It marks a direction of major emphasis in our own future research as now 
planned, One way of putting the matter is this. If m labie 3, for 
example , the dominant variables had turned out to be magnitude variables 
then a less significant first step would have been taken, because anyone 
would iamediately ask what characteristics of the processing done internally 
by the students made these magnitude variables so significant. In postu- 
lating process variables and being able to establish their direct impor- 
tance, we have already been able to mpve past this first step. Now our 
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OBTAINED PROPORTION OF ERRORS 




Figure 9 * 



Scatter plot of observed versus predicted errors on 
fourth-grade addition « 
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central problem is to acquire a better understanding of these variables 
and to use this understanding to develop better predictive models. 

All the analyses reported in this paper have been concerned with 
mean data averaged over individual student performance. Moreover, when 
dealing with data from different age groups, no attempt has been made to 
estimate parameters that would reflect the course of developmental change 
in the performance of arithmetical tasks. Systematic amplification in 
both these directions — taking account of individual differences and devel- 
opmental processes— is relatively straightforward although technically 
arduous for all the models we have considered. A disadvantage of the 
data reported in this paper is that the number of students working at 
any given level and grade was not large. A main objective of the imme- 
diate future is to increase considerably the number of students involved 
in order to provide the quantity of data required for meaningful inferences 
about individual differences or developmental processes. 

Finally, because the data reported here were actually collected in 
an ordinary classroom setting augmented by a computer- controlled terminal, 
and because the data are about performance on standard arithmetical problems, 
it is natural to ask what are implications of our various predictive 
analyses for the teaching of arithmetic. Independent of making any posi- 
tive remarks on this point, we want to underscore the preliminary value 
of our findings. A great deal of more refined analysis with data from 
larger numbers of students is needed to support any definitive pedagogj-cal 
recommendations. Keeping in mind this explicit reservation, we do feel 
that the results that are most intriguing from a pedagogical standpoint 
are the ones reported at the end of the last section on the ability of 



hh 



OBSERVED LATENCY (SECONDS) 



!*U''«a?TeSH*S 



. .fv. 





PREDICTED LATENCY (SECONDS) 



Figure 10. Scatter plot of observed versus predicted 

success-latencies for fourth-grade addition. 
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the memory variable alone to offer a fairly adequate account of the 
observed data. From the way this variable was defined in the theoretical 
section i It should be evident that we can identify some specific points 
to emphasize in teaching multi-digit addition and subtraction. However, 
we leave for another time and place the taking of this ej^licit pedagogical 
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Footnotes 






^o take care of the case when the observed p. is either 0 or 1, 



-.1- - r% , 



use one xoxa.ow’Jing liransiormxiion 



, ^ ]-°s - 



l) for p. = 0 
■^1 



2n. - 1 ^i ^ » 



where n^ = the total niMber of subjects responding to item i. The 
exact form of this transformation is not important. 



The number of subjects or students shown in the various tables is 

always an approximation, with the exact number varying slightly from 

« 

day to day. 



3 

For reasons mentioned below, the first problem was deleted from 
each drill, leaving 19 problems per drill. The number of different- daily 
drills in an analysis can be calculated by dividing the number of problems 
by 19. 
k 

There are some repetitions of problems in Table I9, but all such 
repetitions occurred on different days. 



